Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Christoph, Reopening thread for discussion so that I can proceed to generate next patch. Recap: rdma cgroup controller in the patch defines the framework so that RDMA subsystem can define the resources. This is similar to at least two functionality provided by core kernel. (a) Block elevator

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Christoph, Reopening thread for discussion so that I can proceed to generate next patch. Recap: rdma cgroup controller in the patch defines the framework so that RDMA subsystem can define the resources. This is similar to at least two functionality provided by core kernel. (a) Block elevator

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Christoph, I was on travel. Sorry for the late inline response and question. Parav On Tue, Apr 5, 2016 at 10:57 PM, Christoph Hellwig wrote: > On Tue, Apr 05, 2016 at 05:55:26AM -0700, Parav Pandit wrote: >> Just because we add one more rdma resource, we need to ask

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Christoph, I was on travel. Sorry for the late inline response and question. Parav On Tue, Apr 5, 2016 at 10:57 PM, Christoph Hellwig wrote: > On Tue, Apr 05, 2016 at 05:55:26AM -0700, Parav Pandit wrote: >> Just because we add one more rdma resource, we need to ask someone to >> upgrade

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Tue, Apr 05, 2016 at 05:55:26AM -0700, Parav Pandit wrote: > Just because we add one more rdma resource, we need to ask someone to > upgrade kernel? Yes. Just like when you need any other core kernel resource.

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Tue, Apr 05, 2016 at 05:55:26AM -0700, Parav Pandit wrote: > Just because we add one more rdma resource, we need to ask someone to > upgrade kernel? Yes. Just like when you need any other core kernel resource.

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Tue, Apr 05, 2016 at 05:55:26AM -0700, Parav Pandit wrote: > Hi Christoph, > > On Tue, Apr 5, 2016 at 5:42 AM, Christoph Hellwig wrote: > > On Tue, Apr 05, 2016 at 05:39:21AM -0700, Parav Pandit wrote: > >> I am not really trying to address OFED issues here. I am sure you

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Tue, Apr 05, 2016 at 05:55:26AM -0700, Parav Pandit wrote: > Hi Christoph, > > On Tue, Apr 5, 2016 at 5:42 AM, Christoph Hellwig wrote: > > On Tue, Apr 05, 2016 at 05:39:21AM -0700, Parav Pandit wrote: > >> I am not really trying to address OFED issues here. I am sure you > >> understand that

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hello, Parav. On Tue, Apr 05, 2016 at 07:25:11AM -0700, Parav Pandit wrote: > > As it is a fairly isolated domain, to certain extent, it could be okay > > to let it go. At the same time, I know that these enterprise things > > tend to go completely wayward and am worried about individual drivers

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hello, Parav. On Tue, Apr 05, 2016 at 07:25:11AM -0700, Parav Pandit wrote: > > As it is a fairly isolated domain, to certain extent, it could be okay > > to let it go. At the same time, I know that these enterprise things > > tend to go completely wayward and am worried about individual drivers

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Tue, Apr 5, 2016 at 7:01 AM, Tejun Heo wrote: > Hello, Parav. > > On Mon, Apr 04, 2016 at 07:22:38PM -0700, Parav Pandit wrote: >> > Is it actually customary to have rdma core module updated more >> > frequently separate from the kernel? Out-of-tree modules being >> > updated

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Tue, Apr 5, 2016 at 7:01 AM, Tejun Heo wrote: > Hello, Parav. > > On Mon, Apr 04, 2016 at 07:22:38PM -0700, Parav Pandit wrote: >> > Is it actually customary to have rdma core module updated more >> > frequently separate from the kernel? Out-of-tree modules being >> > updated separately

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hello, Parav. On Mon, Apr 04, 2016 at 07:22:38PM -0700, Parav Pandit wrote: > > Is it actually customary to have rdma core module updated more > > frequently separate from the kernel? Out-of-tree modules being > > updated separately happens quite a bit but this is an in-kernel > > module, which

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hello, Parav. On Mon, Apr 04, 2016 at 07:22:38PM -0700, Parav Pandit wrote: > > Is it actually customary to have rdma core module updated more > > frequently separate from the kernel? Out-of-tree modules being > > updated separately happens quite a bit but this is an in-kernel > > module, which

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Tue, Apr 5, 2016 at 7:07 AM, Tejun Heo wrote: > Just one more thing. > > On Tue, Apr 05, 2016 at 10:01:07AM -0400, Tejun Heo wrote: > ... >> > pool_info in spin lock context, made me allocate memory to get all >> > values upfront through allocation. >> > Now that the lock is

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Tue, Apr 5, 2016 at 7:07 AM, Tejun Heo wrote: > Just one more thing. > > On Tue, Apr 05, 2016 at 10:01:07AM -0400, Tejun Heo wrote: > ... >> > pool_info in spin lock context, made me allocate memory to get all >> > values upfront through allocation. >> > Now that the lock is going away, I can

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Just one more thing. On Tue, Apr 05, 2016 at 10:01:07AM -0400, Tejun Heo wrote: ... > > pool_info in spin lock context, made me allocate memory to get all > > values upfront through allocation. > > Now that the lock is going away, I can do what you have described above. So, this might be okay

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Just one more thing. On Tue, Apr 05, 2016 at 10:01:07AM -0400, Tejun Heo wrote: ... > > pool_info in spin lock context, made me allocate memory to get all > > values upfront through allocation. > > Now that the lock is going away, I can do what you have described above. So, this might be okay

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Christoph, On Tue, Apr 5, 2016 at 5:42 AM, Christoph Hellwig wrote: > On Tue, Apr 05, 2016 at 05:39:21AM -0700, Parav Pandit wrote: >> I am not really trying to address OFED issues here. I am sure you >> understand that if ib_core.ko kernel module is in-kernel module than,

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Christoph, On Tue, Apr 5, 2016 at 5:42 AM, Christoph Hellwig wrote: > On Tue, Apr 05, 2016 at 05:39:21AM -0700, Parav Pandit wrote: >> I am not really trying to address OFED issues here. I am sure you >> understand that if ib_core.ko kernel module is in-kernel module than, >> for all the

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Tue, Apr 05, 2016 at 05:39:21AM -0700, Parav Pandit wrote: > I am not really trying to address OFED issues here. I am sure you > understand that if ib_core.ko kernel module is in-kernel module than, > for all the fixes/enhancements that goes to it would require people to > upgrade to newer

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Tue, Apr 05, 2016 at 05:39:21AM -0700, Parav Pandit wrote: > I am not really trying to address OFED issues here. I am sure you > understand that if ib_core.ko kernel module is in-kernel module than, > for all the fixes/enhancements that goes to it would require people to > upgrade to newer

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Christoph, On Tue, Apr 5, 2016 at 2:06 AM, Christoph Hellwig wrote: > On Mon, Apr 04, 2016 at 09:25:04PM -0400, Tejun Heo wrote: >> Is it actually customary to have rdma core module updated more >> frequently separate from the kernel? Out-of-tree modules being >> updated

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Christoph, On Tue, Apr 5, 2016 at 2:06 AM, Christoph Hellwig wrote: > On Mon, Apr 04, 2016 at 09:25:04PM -0400, Tejun Heo wrote: >> Is it actually customary to have rdma core module updated more >> frequently separate from the kernel? Out-of-tree modules being >> updated separately happens

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Mon, Apr 04, 2016 at 09:25:04PM -0400, Tejun Heo wrote: > Is it actually customary to have rdma core module updated more > frequently separate from the kernel? Out-of-tree modules being > updated separately happens quite a bit but this is an in-kernel > module, which usually is tightly coupled

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

On Mon, Apr 04, 2016 at 09:25:04PM -0400, Tejun Heo wrote: > Is it actually customary to have rdma core module updated more > frequently separate from the kernel? Out-of-tree modules being > updated separately happens quite a bit but this is an in-kernel > module, which usually is tightly coupled

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Tejun, On Mon, Apr 4, 2016 at 6:25 PM, Tejun Heo wrote: > Hello, > > On Mon, Apr 04, 2016 at 03:50:54PM -0700, Parav Pandit wrote: >> 2nd advantage is, it allows avoiding lot of rework, in bundling kernel >> modules with different kernel versions. So it is certainly valuable

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Tejun, On Mon, Apr 4, 2016 at 6:25 PM, Tejun Heo wrote: > Hello, > > On Mon, Apr 04, 2016 at 03:50:54PM -0700, Parav Pandit wrote: >> 2nd advantage is, it allows avoiding lot of rework, in bundling kernel >> modules with different kernel versions. So it is certainly valuable >> feature with

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hello, On Mon, Apr 04, 2016 at 03:50:54PM -0700, Parav Pandit wrote: > 2nd advantage is, it allows avoiding lot of rework, in bundling kernel > modules with different kernel versions. So it is certainly valuable > feature with almost nil complexity cost of code or memory that solves > two

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hello, On Mon, Apr 04, 2016 at 03:50:54PM -0700, Parav Pandit wrote: > 2nd advantage is, it allows avoiding lot of rework, in bundling kernel > modules with different kernel versions. So it is certainly valuable > feature with almost nil complexity cost of code or memory that solves > two

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Tejun, On Mon, Apr 4, 2016 at 12:36 PM, Tejun Heo wrote: > Hello, Parav. > > Sorry about the delay. > Thanks for the review. Comments and answers inline. >> +struct rdma_cgroup { >> + struct cgroup_subsys_state css; >> + >> + /* protects resource pool list */ >>

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hi Tejun, On Mon, Apr 4, 2016 at 12:36 PM, Tejun Heo wrote: > Hello, Parav. > > Sorry about the delay. > Thanks for the review. Comments and answers inline. >> +struct rdma_cgroup { >> + struct cgroup_subsys_state css; >> + >> + /* protects resource pool list */ >> + spinlock_t

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hello, Parav. Sorry about the delay. > +struct rdma_cgroup { > + struct cgroup_subsys_state css; > + > + /* protects resource pool list */ > + spinlock_t rpool_list_lock; Is this lock still being used? > + /* > + * head to keep track of all

Re: [PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Hello, Parav. Sorry about the delay. > +struct rdma_cgroup { > + struct cgroup_subsys_state css; > + > + /* protects resource pool list */ > + spinlock_t rpool_list_lock; Is this lock still being used? > + /* > + * head to keep track of all

[PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Added rdma cgroup controller that does accounting, limit enforcement on rdma/IB verbs and hw resources. Added rdma cgroup header file which defines its APIs to perform charing/uncharing functionality and device registration which will participate in controller functions of accounting and limit

[PATCHv10 1/3] rdmacg: Added rdma cgroup controller

Added rdma cgroup controller that does accounting, limit enforcement on rdma/IB verbs and hw resources. Added rdma cgroup header file which defines its APIs to perform charing/uncharing functionality and device registration which will participate in controller functions of accounting and limit