Re: Major Roadblocks - real life use cases
I have a couple of new bits and pieces coming to the data charms soonish so if no one beats me to it we can take a look at Hadoop storage also. Tom On 29 May 2018 at 16:50:22, James Beedy (jamesbe...@gmail.com) wrote: Drew, Thanks for the response. I've been dabbling in these areas. Looks like I may just need to use a combination of a few different methods here. ~James On Tue, May 29, 2018 at 8:45 AM, Drew Freiberger < drew.freiber...@canonical.com> wrote: > On Mon, May 28, 2018 at 10:23:21PM -0700, James Beedy wrote: > >> I want to shed some light on a few blockers for me right now. >> >> 2) Maas needs better support for 3rd party drivers. >> * Getting my Mellanox drivers hooked up at commissioning so maas >> recognizes the 40Gb interfaces is taking me a few weeks now. Supporting >> user defined 3rd party driver should be a primary supported capability of >> MAAS. >> * how are people doing Hadoop,and Ceph and Openstack without this, >> possibly someone knows something I don’t know here? >> > > Hello James, > > I believe that MAAS 2.x's nodes-scripts (Hardware Testing and > Commissioning scripts) might be helpful for any custom code or > drivers you might want to inject in the commissioning process. > > Here is a link to the guide for these scripts. I might suggest > the cript example for "Configure HPA" might provide a good template for > how you might want to inject a Mellanox driver/config into your build. > > https://docs.maas.io/2.3/en/nodes-scripts > > The MAAS server has a web server serving out a docroot from > /var/www/htdocs where you can drop any files you might want to curl/wget > from your scripts. I believe there are environment variables for the > preseeds for the IP of the MAAS server for your URL. > > You might also notice that you can have the script automatically install > packages from any of apt, snap, or URL. You can even tag the script to > automatically run on hardware with a specific PCI ID. with the > for_hardware metadata field. > > Documentation on managing these scripts via the CLI is here: > > https://docs.maas.io/2.3/en/nodes-scripts-cli > > In MAAS 1 environments, one would have to update the curtin preseeds > manually in /etc/maas/preseeds to inject scripts of this type. > > I do not know if the commissioning scripts also happen at deployment > time (doubtful), but hopefully the install of your chosen OS will > include the drivers needed, or you've found that you can add an Ubuntu > package repository or PPA that contains your packages to be installed > per https://docs.maas.io/2.3/en/manage-repos. > > I hope this information helps. > > Sincerely, > -Drew > > Any feedback would be greatly appreciated. >> >> Thanks >> >> >> >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm >> an/listinfo/juju >> > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju -- Spicule Limited is registered in England & Wales. Company Number: 09954122. Registered office: First Floor, Telecom House, 125-135 Preston Road, Brighton, England, BN1 6AF. VAT No. 251478891. All engagements are subject to Spicule Terms and Conditions of Business. This email and its contents are intended solely for the individual to whom it is addressed and may contain information that is confidential, privileged or otherwise protected from disclosure, distributing or copying. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Spicule Limited. The company accepts no liability for any damage caused by any virus transmitted by this email. If you have received this message in error, please notify us immediately by reply email before deleting it from your system. Service of legal notice cannot be effected on Spicule Limited by email. -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Major Roadblocks - real life use cases
Drew, Thanks for the response. I've been dabbling in these areas. Looks like I may just need to use a combination of a few different methods here. ~James On Tue, May 29, 2018 at 8:45 AM, Drew Freiberger < drew.freiber...@canonical.com> wrote: > On Mon, May 28, 2018 at 10:23:21PM -0700, James Beedy wrote: > >> I want to shed some light on a few blockers for me right now. >> >> 2) Maas needs better support for 3rd party drivers. >> * Getting my Mellanox drivers hooked up at commissioning so maas >> recognizes the 40Gb interfaces is taking me a few weeks now. Supporting >> user defined 3rd party driver should be a primary supported capability of >> MAAS. >> * how are people doing Hadoop,and Ceph and Openstack without this, >> possibly someone knows something I don’t know here? >> > > Hello James, > > I believe that MAAS 2.x's nodes-scripts (Hardware Testing and > Commissioning scripts) might be helpful for any custom code or > drivers you might want to inject in the commissioning process. > > Here is a link to the guide for these scripts. I might suggest > the cript example for "Configure HPA" might provide a good template for > how you might want to inject a Mellanox driver/config into your build. > > https://docs.maas.io/2.3/en/nodes-scripts > > The MAAS server has a web server serving out a docroot from > /var/www/htdocs where you can drop any files you might want to curl/wget > from your scripts. I believe there are environment variables for the > preseeds for the IP of the MAAS server for your URL. > > You might also notice that you can have the script automatically install > packages from any of apt, snap, or URL. You can even tag the script to > automatically run on hardware with a specific PCI ID. with the > for_hardware metadata field. > > Documentation on managing these scripts via the CLI is here: > > https://docs.maas.io/2.3/en/nodes-scripts-cli > > In MAAS 1 environments, one would have to update the curtin preseeds > manually in /etc/maas/preseeds to inject scripts of this type. > > I do not know if the commissioning scripts also happen at deployment > time (doubtful), but hopefully the install of your chosen OS will > include the drivers needed, or you've found that you can add an Ubuntu > package repository or PPA that contains your packages to be installed > per https://docs.maas.io/2.3/en/manage-repos. > > I hope this information helps. > > Sincerely, > -Drew > > Any feedback would be greatly appreciated. >> >> Thanks >> >> >> >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailm >> an/listinfo/juju >> > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Major Roadblocks - real life use cases
On Mon, May 28, 2018 at 10:23:21PM -0700, James Beedy wrote: I want to shed some light on a few blockers for me right now. 2) Maas needs better support for 3rd party drivers. * Getting my Mellanox drivers hooked up at commissioning so maas recognizes the 40Gb interfaces is taking me a few weeks now. Supporting user defined 3rd party driver should be a primary supported capability of MAAS. * how are people doing Hadoop,and Ceph and Openstack without this, possibly someone knows something I don’t know here? Hello James, I believe that MAAS 2.x's nodes-scripts (Hardware Testing and Commissioning scripts) might be helpful for any custom code or drivers you might want to inject in the commissioning process. Here is a link to the guide for these scripts. I might suggest the cript example for "Configure HPA" might provide a good template for how you might want to inject a Mellanox driver/config into your build. https://docs.maas.io/2.3/en/nodes-scripts The MAAS server has a web server serving out a docroot from /var/www/htdocs where you can drop any files you might want to curl/wget from your scripts. I believe there are environment variables for the preseeds for the IP of the MAAS server for your URL. You might also notice that you can have the script automatically install packages from any of apt, snap, or URL. You can even tag the script to automatically run on hardware with a specific PCI ID. with the for_hardware metadata field. Documentation on managing these scripts via the CLI is here: https://docs.maas.io/2.3/en/nodes-scripts-cli In MAAS 1 environments, one would have to update the curtin preseeds manually in /etc/maas/preseeds to inject scripts of this type. I do not know if the commissioning scripts also happen at deployment time (doubtful), but hopefully the install of your chosen OS will include the drivers needed, or you've found that you can add an Ubuntu package repository or PPA that contains your packages to be installed per https://docs.maas.io/2.3/en/manage-repos. I hope this information helps. Sincerely, -Drew Any feedback would be greatly appreciated. Thanks -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju signature.asc Description: PGP signature -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Major Roadblocks - real life use cases
I want to shed some light on a few blockers for me right now. 1) I am having a hard time actually using Hadoop charms for Hadoop deploys on metal due to 2 reasons: * Maas support for 3rd party drivers a pain * Hadoop charms don’t have storage support 2) I am having difficulty deploying Ceph and Neutron due to: * maas support for 3rd party drivers a pain Here are my two asks, hopefully you will find them as pertinent as I do. 1) Hadoop charms need storage support * hdfs * cephfs Currently without support for storage, Hadoop slaves just use a local directory on / for HDFS. This is entirely illegitimate for any real batch processing use case using HDFS (something I’m trying to do right now but can’t use juju Hadoop to do because of the lack of support for storage). 2) Maas needs better support for 3rd party drivers. * Getting my Mellanox drivers hooked up at commissioning so maas recognizes the 40Gb interfaces is taking me a few weeks now. Supporting user defined 3rd party driver should be a primary supported capability of MAAS. * how are people doing Hadoop,and Ceph and Openstack without this, possibly someone knows something I don’t know here? Any feedback would be greatly appreciated. Thanks -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
