On Fri, Oct 27, 2017 at 4:24 PM, Francois Ozog <francois.o...@linaro.org> wrote:
> > Le ven. 27 oct. 2017 à 23:05, Honnappa Nagarahalli < > honnappa.nagaraha...@linaro.org> a écrit : > >> On 27 October 2017 at 13:35, Bill Fischofer <bill.fischo...@linaro.org> >> wrote: >> >>> >>> >>> On Fri, Oct 27, 2017 at 10:45 AM, Francois Ozog < >>> francois.o...@linaro.org> wrote: >>> >>>> >>>> Le ven. 27 oct. 2017 à 17:17, Bill Fischofer <bill.fischo...@linaro.org> >>>> a écrit : >>>> >>>>> The problem with scanning, especially in a VNF environment, is that >>>>> (a) the application probably isn't authorized to to that >>>>> >>>> >>>> nothing prevents scanning what is available for n the vm. "Scale up" >>>> Events are triggered when increasing resources (memory cpus Ethernet >>>> ports...) >>>> >>>> and (b) the application certainly doesn't have real visibility into >>>>> what the actual device topology is. >>>>> >>>> >>>> Qemu allows to select if the VM has visibility of a real nuns topology >>>> or a virtua one >>>> >>> >>> Qemu may itself be running under a hypervisor and have limited >>> visibility as to the real HW configuration. The whole point of >>> virtualization/containerization is to limit what the VM/application can >>> see and do to provide management controls over isolation and portability. >>> >> >> In this case, the 'platform' refers to what is available in the VM. There >> is no need to know what is available in the underlying hardware. >> > > Yes. That said there is a corner case with FPGA. When à VM loads a > bitstream to an FPGA slice , it is expected that a new PCI device to deal > with the configured FPGA is dynamically passed to the VM. The VM needs to > know the underlying FPGA chip to load the proper bitstream (altera Xilinx > microsemi...). > In that case either the VM has access to the "unconfigured" FPGA (which is still capable of identifying itself while unconfigured) and writes this directly, or else this is done at the host level and the result is made available to the VM. > >>> >>>> >>>> It only knows what it "needs to know" (as determined by higher-level >>>>> functions) so there's no point trying to second-guess that. Applications >>>>> will be told "use these devices" and that should be our design assumption >>>>> in this support. >>>>> >>>> >>>> Vnf manager will know precisely the host vision of things (use this >>>> vhostuser interface) but don't don't know how it may end up being named in >>>> guest . This is a key problem that has been identified by British telecom . >>>> >>> >>> Sounds like we should follow whatever recommended solution OPNFV comes >>> up with in this area. In any event, that solution will be outside of ODP >>> and would still result in the application being told what devices to use >>> with names that are meaningful in the environment it's running in. >>> >>> >>>> >>>>> On Fri, Oct 27, 2017 at 10:10 AM, Honnappa Nagarahalli < >>>>> honnappa.nagaraha...@linaro.org> wrote: >>>>> >>>>>> >>>>>> >>>>>> On 27 October 2017 at 09:50, Bill Fischofer < >>>>>> bill.fischo...@linaro.org> wrote: >>>>>> >>>>>>> ODP 2.0 assumes Linux system services are available so the question >>>>>>> of how to operate in bare metal environments is a separate one and up to >>>>>>> those ODP implementations. Again the application will provide a >>>>>>> sufficiently-qualified device name string to identify which device it >>>>>>> wants >>>>>>> to open in an unambiguous manner. How it does that is again outside the >>>>>>> scope of ODP so this isn't something we need to worry about. All ODP >>>>>>> needs >>>>>>> to do is be able to identify which driver needs to be loaded and passed >>>>>>> the >>>>>>> rest of the device name and then that driver handles the rest. >>>>>>> >>>>>>> By baremetal, I meant host with Linux OS. >>>>>> I agree, it is applications responsibility to provide the device >>>>>> string, how it does that is outside the scope of ODP. >>>>>> We will continue the discussion on the slides. But, in the meanwhile, >>>>>> I am thinking of possible design if we want to avoid complete scanning of >>>>>> the platform during initialization. In the current packet I/O framework, >>>>>> all the function pointers of the driver are known before odp_pktio_open >>>>>> API >>>>>> is called. If we have to stick to this design, the drivers for the PCI >>>>>> device should have registered their functions with the packet IO >>>>>> framework >>>>>> before odp_pktio_open is called. >>>>>> >>>>>> >>>>>>> On Fri, Oct 27, 2017 at 2:36 AM, Francois Ozog < >>>>>>> francois.o...@linaro.org> wrote: >>>>>>> >>>>>>>> Well, we do not need to scan all the platform because the >>>>>>>> pktio_open contains enough information to target the right device. >>>>>>>> This is almost true as we need to have an additional "selector" for >>>>>>>> the port on multiport NICs that are controlled by a single pci ID. >>>>>>>> <enumerator>:<enumerator specific string>:<port> or something like >>>>>>>> that. >>>>>>>> >>>>>>>> All ports may not be typically handled by ODP. For instance >>>>>>>> management network will most certainly be a native Linux one. >>>>>>>> >>>>>>>> Dpaa2 bus may impose us to full scan if we have to go the full >>>>>>>> driver route (vfio-pci) but this will not be necessary if we can have >>>>>>>> the >>>>>>>> net-mdev. In that case, the dpaa2 bus can be handled the same way as >>>>>>>> pci. >>>>>>>> >>>>>>>> Dynamic bus events (hot plug) are a nice to have and may be dropped >>>>>>>> from coding yet we can talk about it. and when it come to this, this >>>>>>>> is not >>>>>>>> about dealing with the bus controllers directly but tapping into Linux >>>>>>>> event framework. >>>>>>>> >>>>>>>> I know we can simplify things and I am very flexible in what we can >>>>>>>> decide not to do. That said I have been dealing with operational >>>>>>>> issues on >>>>>>>> that very topic since 2006 when I designed my first kernel based fast >>>>>>>> packet IO.... So there will be topics where I'll push hard to explain >>>>>>>> (not >>>>>>>> impose) why we should go a harder route. This slows things down but I >>>>>>>> think >>>>>>>> this is worth it. >>>>>>>> >>>>>>>> FF >>>>>>>> >>>>>>>> Le ven. 27 oct. 2017 à 06:23, Honnappa Nagarahalli < >>>>>>>> honnappa.nagaraha...@linaro.org> a écrit : >>>>>>>> >>>>>>>>> On 26 October 2017 at 16:34, Bill Fischofer < >>>>>>>>> bill.fischo...@linaro.org> wrote: >>>>>>>>> > I agree with Maxim. Best to get one or two working drivers and >>>>>>>>> see what else >>>>>>>>> > is needed. The intent here is not for ODP to become another OS, >>>>>>>>> so I'm not >>>>>>>>> > sure why we need to concern ourselves with bus walking and >>>>>>>>> similar arcana. >>>>>>>>> > Linux has already long solved this problem. We should leverage >>>>>>>>> what's there, >>>>>>>>> > not try to reinvent it. >>>>>>>>> > >>>>>>>>> > ODP applications are told what I/O interfaces to use, either >>>>>>>>> through an >>>>>>>>> > application configuration file, command line, or other means >>>>>>>>> outside the >>>>>>>>> > scope of ODP itself. ODP's job is simply to connect applications >>>>>>>>> to these >>>>>>>>> > configured I/O interfaces when they call odp_pktio_open(). The >>>>>>>>> name used for >>>>>>>>> > interfaces is simply a string that we've defined to have the >>>>>>>>> format: >>>>>>>>> > >>>>>>>>> > class: device: other-info-needed-by-driver >>>>>>>>> > >>>>>>>>> > We've defined a number of classes already: >>>>>>>>> > >>>>>>>>> > - loop >>>>>>>>> > - pcap >>>>>>>>> > - ipc >>>>>>>>> > - dpdk >>>>>>>>> > - socket >>>>>>>>> > - socket_mmap >>>>>>>>> > - tap >>>>>>>>> > etc. >>>>>>>>> > >>>>>>>>> > We simply need to define new classes (e.g., ddf, mdev) and the >>>>>>>>> names they >>>>>>>>> > need to take to identify a specific device and the associated >>>>>>>>> driver to use >>>>>>>>> > for operate that device. The driver is then loaded if necessary >>>>>>>>> and its >>>>>>>>> > open() interface is called. >>>>>>>>> > >>>>>>>>> >>>>>>>>> Coincidentally, internally we were discussing this exactly. >>>>>>>>> >>>>>>>>> Why do we need to scan and understand the complete platform during >>>>>>>>> initialization? - I would think, mostly, ODP will run in a platform >>>>>>>>> (baremetal, VM, container) where all the devices are supposed to be >>>>>>>>> used by ODP (i.e. ODP will not run in a platform where it will >>>>>>>>> share >>>>>>>>> the platform with other applications). So why not scan and identify >>>>>>>>> the devices/drivers and create the packet IO ops during >>>>>>>>> initialization. The packet I/O framework assumes that various >>>>>>>>> methods >>>>>>>>> (open, close, send, recv etc) of the device driver are known when >>>>>>>>> odp_pktio_open API is called. None of that has to change. >>>>>>>>> >>>>>>>>> Another solution I can think of is (tweak to reduce the time spent >>>>>>>>> in >>>>>>>>> scanning etc), instead of scanning all the devices, DDF >>>>>>>>> initialization >>>>>>>>> function can be provided with all the ports the user has requested. >>>>>>>>> >>>>>>>>> If the scanning for the device and identifying the driver has to be >>>>>>>>> triggered through the odp_pktio_open API, the current packet IO >>>>>>>>> framework needs to change. >>>>>>>>> >>>>>>>>> > On Thu, Oct 26, 2017 at 3:59 PM, Maxim Uvarov < >>>>>>>>> maxim.uva...@linaro.org> >>>>>>>>> > wrote: >>>>>>>>> >> >>>>>>>>> >> Hello Honnappa, >>>>>>>>> >> >>>>>>>>> >> I think we also need to take a look from bottom. I.e. from >>>>>>>>> exact drivers >>>>>>>>> >> to >>>>>>>>> >> implement. That it will be more clear which interface is needed >>>>>>>>> to be >>>>>>>>> >> created. >>>>>>>>> >> Do you have some list of drivers which needed to be >>>>>>>>> implemented? I.e. with >>>>>>>>> >> pci drivers I think we in a good way, but non pci drivers are >>>>>>>>> under >>>>>>>>> >> question. >>>>>>>>> >> I think we should not over-complicate odp itself with huge >>>>>>>>> discovery and >>>>>>>>> >> numeration of devices. Let's take a look what is the minimal >>>>>>>>> interface to >>>>>>>>> >> support >>>>>>>>> >> devices. >>>>>>>>> >> >>>>>>>>> >> Best regards, >>>>>>>>> >> Maxim. >>>>>>>>> >> >>>>>>>>> >> On 26 October 2017 at 23:35, Honnappa Nagarahalli < >>>>>>>>> >> honnappa.nagaraha...@linaro.org> wrote: >>>>>>>>> >> >>>>>>>>> >> > Hi, >>>>>>>>> >> > Agree, we have taken 2 hours and the progress has been >>>>>>>>> slow. But >>>>>>>>> >> > the discussions have been good and helpful to us at Arm. The >>>>>>>>> goal is >>>>>>>>> >> > to identify the gaps and work items. I am not sure if it has >>>>>>>>> been >>>>>>>>> >> > helpful to others, please let me know. >>>>>>>>> >> > >>>>>>>>> >> > To speed this up, I propose few options below, let me know >>>>>>>>> your opinion: >>>>>>>>> >> > >>>>>>>>> >> > 1) Have 2 additional (along with regular ODP 2.0) calls next >>>>>>>>> week - We >>>>>>>>> >> > can do Tuesday 7:00am and then another on Thursday 7:00am >>>>>>>>> (Austin, TX >>>>>>>>> >> > time, GMT-6, one hour before the regular ODP-2.0) >>>>>>>>> >> > >>>>>>>>> >> > 2) Resolve the pending PRs on emails >>>>>>>>> >> > >>>>>>>>> >> > 3) Discuss the DDF slides on email - Not sure how effective >>>>>>>>> it will be >>>>>>>>> >> > >>>>>>>>> >> > Any other solutions? >>>>>>>>> >> > >>>>>>>>> >> > Thank you, >>>>>>>>> >> > Honnappa >>>>>>>>> >> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> >>>>>>>> -- >>>>>>>> [image: Linaro] <http://www.linaro.org/> >>>>>>>> François-Frédéric Ozog | *Director Linaro Networking Group* >>>>>>>> T: +33.67221.6485 >>>>>>>> francois.o...@linaro.org | Skype: ffozog >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> -- >>>> [image: Linaro] <http://www.linaro.org/> >>>> François-Frédéric Ozog | *Director Linaro Networking Group* >>>> T: +33.67221.6485 >>>> francois.o...@linaro.org | Skype: ffozog >>>> >>>> >>> -- > [image: Linaro] <http://www.linaro.org/> > François-Frédéric Ozog | *Director Linaro Networking Group* > T: +33.67221.6485 > francois.o...@linaro.org | Skype: ffozog > >
