Re: Charm layers & Windows
On Mon, Apr 4, 2016 at 10:42 PM Marco Ceppi wrote: > There are two things that need to be done. The first, we need the reactive > framework to be ported to powershell - that way we can have charms written > in powershell and compiled as such. I know the cloud base folks poked at > that a bit in Gent during the Summit but I haven't heard much from there. > > The second, is two base layers. The first is a powershell base layer so > you get the awesome powerhshell helpers cloudbase has created (like the > python charm helpers). That way native power shell layers can be written. > The second is to create a python-windows base layer, this would be the > basic layer and then the necessary methods to install Python on the windows > machine so that python layers work properly. > > Some of this we can pilot ourselves, (mostly the python-windows layer) - > some of the team is sprinting so I'll add that as a stretch goal. The > powershell native features we'll need help and I admit I've done a terrible > job keeping up with the cloudbase folks who have been invaluable as a > windows + juju resource thus far. > Thanks, Marco. FWIW, I had imagined an MVP just as Stuart described: add the Windows bootstrap scripts (install.ps1|bat|cmd, etc.), which should just need to install Python and then defer to the reactive framework. Going full Powershell support sounds ideal, but not what I'm after. Cheers, Andrew > Marco > > On Mon, Apr 4, 2016 at 7:46 AM Rick Harding > wrote: > >> I know that Gabriel and some of the CloudBase folks seemed interested in >> layers and possibly some tooling with powershell. I'm not sure how far that >> went but I thought they were experimenting during the charmer's summit. >> That would help with a charm build on windows, but not for some common code >> between both operating systems. >> >> An interesting thing is how much setup and how ootb the Ubuntu on Windows >> needs. If it's working out of the box, it might be an interesting move for >> us and our tools that Windows users could get a Linux experience. I guess >> that it won't be ideal though as I'm not sure what the server side plans >> around that work is. >> >> On Mon, Apr 4, 2016 at 3:18 AM Andrew Wilkins < >> andrew.wilk...@canonical.com> wrote: >> >>> Hi, >>> >>> I would like to write a charm that should be mostly identical on Windows >>> and Linux, so I think it would make sense to have common code in the form >>> of a layer. >>> >>> Is anyone working on getting "charm build", layers, and friends to work >>> with Windows workloads? If not, I may look into it myself. >>> >>> Cheers, >>> Andrew >>> -- >>> Juju mailing list >>> Juju@lists.ubuntu.com >>> Modify settings or unsubscribe at: >>> https://lists.ubuntu.com/mailman/listinfo/juju >>> >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju >> > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Action return value too long for commandline
Hi Stuart Thanks for this explanation. 16MB is definitely a step in the right direction. However, in the future I'll need to transfer even bigger files. Is there a possibility for using resources for this in the future? It would be great if an action could upload a file to the resources server so the admin can download it from there. Kind regards Merlijn Sebrechts 2016-04-03 19:34 GMT-07:00 Stuart Bishop : > On 4 April 2016 at 02:00, Merlijn Sebrechts > wrote: > > Hi all > > > > > > The apache-kafka charm has an action "read-topic" that can return a lot > of > > data. Sometimes, this data is too long to be passed to `action-set` by > > commandline. You get the following error: > > > > Traceback (most recent call last): > > File "actions/read-topic", line 36, in > > hookenv.action_set({'output': output}) > > File > > "/usr/local/lib/python2.7/dist-packages/charmhelpers/core/hookenv.py", > line > > 615, in action_set > > subprocess.check_call(cmd) > > File "/usr/lib/python2.7/subprocess.py", line 535, in check_call > > retcode = call(*popenargs, **kwargs) > > File "/usr/lib/python2.7/subprocess.py", line 522, in call > > return Popen(*popenargs, **kwargs).wait() > > File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ > > errread, errwrite) > > File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child > > raise child_exception > > OSError: [Errno 7] Argument list too long > > > > > > Is there any way to pass a file to action-set? > > This bug affects many of the Juju tools causing charms to fail at > scale. https://bugs.launchpad.net/juju-core/+bug/1437366 > (relation-set) is the only one I know of that has been fixed. > https://bugs.launchpad.net/juju-core/+bug/1274460 (juju-log) is still > open. leader-set also fails now I think of it, but I haven't tripped > over that one (it limits scalability the way I'm using leadershp > settings, but I should be able to squeeze out several hundred units). > Maybe we can use the opportunity to fix quoting and encoding issues. > > I suspect if you can get past command line length limitations, I > suspect the next glass ceiling is a 16MB document size limit in > MongoDB (which is large enough to not need fixing?) > > -- > Stuart Bishop > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Charm layers & Windows
There are two things that need to be done. The first, we need the reactive framework to be ported to powershell - that way we can have charms written in powershell and compiled as such. I know the cloud base folks poked at that a bit in Gent during the Summit but I haven't heard much from there. The second, is two base layers. The first is a powershell base layer so you get the awesome powerhshell helpers cloudbase has created (like the python charm helpers). That way native power shell layers can be written. The second is to create a python-windows base layer, this would be the basic layer and then the necessary methods to install Python on the windows machine so that python layers work properly. Some of this we can pilot ourselves, (mostly the python-windows layer) - some of the team is sprinting so I'll add that as a stretch goal. The powershell native features we'll need help and I admit I've done a terrible job keeping up with the cloudbase folks who have been invaluable as a windows + juju resource thus far. Marco On Mon, Apr 4, 2016 at 7:46 AM Rick Harding wrote: > I know that Gabriel and some of the CloudBase folks seemed interested in > layers and possibly some tooling with powershell. I'm not sure how far that > went but I thought they were experimenting during the charmer's summit. > That would help with a charm build on windows, but not for some common code > between both operating systems. > > An interesting thing is how much setup and how ootb the Ubuntu on Windows > needs. If it's working out of the box, it might be an interesting move for > us and our tools that Windows users could get a Linux experience. I guess > that it won't be ideal though as I'm not sure what the server side plans > around that work is. > > On Mon, Apr 4, 2016 at 3:18 AM Andrew Wilkins < > andrew.wilk...@canonical.com> wrote: > >> Hi, >> >> I would like to write a charm that should be mostly identical on Windows >> and Linux, so I think it would make sense to have common code in the form >> of a layer. >> >> Is anyone working on getting "charm build", layers, and friends to work >> with Windows workloads? If not, I may look into it myself. >> >> Cheers, >> Andrew >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju >> > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Charm layers & Windows
BTW, I don't believe this is true. I didn't work on series in metadata, but I skimmed the code and didn't see anything doing checks for OS consistency across the stated series. I just tried it with a charm labeled as trusty & win10 in the metadata.yaml and it worked fine deploying to trusty. On Mon, Apr 4, 2016 at 9:48 AM Stuart Bishop wrote: > > You will need two root layers for the two charms, one for Ubuntu and > one for Windows, since you can't list both Windows and Ubuntu releases > as supported series in the same metadata.yaml. > > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Charm layers & Windows
At least... according to the core code, but it occurs to me I may well be mistaken about what's allowed by the charm authoring tools and/or charmstore. On Mon, Apr 4, 2016 at 10:16 AM Nate Finch wrote: > BTW, I don't believe this is true. I didn't work on series in metadata, > but I skimmed the code and didn't see anything doing checks for OS > consistency across the stated series. I just tried it with a charm labeled > as trusty & win10 in the metadata.yaml and it worked fine deploying to > trusty. > > On Mon, Apr 4, 2016 at 9:48 AM Stuart Bishop > wrote: > >> >> You will need two root layers for the two charms, one for Ubuntu and >> one for Windows, since you can't list both Windows and Ubuntu releases >> as supported series in the same metadata.yaml. >> >> -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Charm layers & Windows
On 4 April 2016 at 14:17, Andrew Wilkins wrote: > Hi, > > I would like to write a charm that should be mostly identical on Windows and > Linux, so I think it would make sense to have common code in the form of a > layer. > > Is anyone working on getting "charm build", layers, and friends to work with > Windows workloads? If not, I may look into it myself. I think you could do it right now if you added explicit powershell hook stubs that a) ensure Python is installed on the path and b) kick off the Python hooks. https://github.com/juju/charm-tools/issues/98 is about having 'charm build' generate the required hook stubs from layer.yaml instead of embedding them, which will help here in the future. You will need two root layers for the two charms, one for Ubuntu and one for Windows, since you can't list both Windows and Ubuntu releases as supported series in the same metadata.yaml. A source of potential problems is charmhelpers as it is not tested under Windows and there are likely linuxisms burried in there. charms.reactive doesn't need much of it though, so it is likely to be fine. I haven't seen anything obviously platform dependent apart from the obvious stuff like wrappers around apt. There might be some linux specific imports at the top level that need to be moved to import on demand (import apt, import distro_info etc). -- Stuart Bishop -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Charm layers & Windows
>From what I've read, there's no server-side story to Ubuntu on Windows. It's purely desktop, and in fact, only installs on desktop versions of Windows 10. That could help with tooling for charm authors, but obviously the charm code itself still needs to run on vanilla Windows. The main tricky part I see is that the charm code for Windows expects hooks to be .exe, .ps1, .cmd, or .bat. You can't have a hook with no extension on Windows, and on Linux we expect it to be a command with no extension. That being said, there's nothing that says you can't have both an 'install.ps1' hook file for Windows and an 'install' hook file for Linux in the same charm. The code will do the right thing and use the right hook for each OS. On Mon, Apr 4, 2016 at 7:46 AM Rick Harding wrote: > I know that Gabriel and some of the CloudBase folks seemed interested in > layers and possibly some tooling with powershell. I'm not sure how far that > went but I thought they were experimenting during the charmer's summit. > That would help with a charm build on windows, but not for some common code > between both operating systems. > > An interesting thing is how much setup and how ootb the Ubuntu on Windows > needs. If it's working out of the box, it might be an interesting move for > us and our tools that Windows users could get a Linux experience. I guess > that it won't be ideal though as I'm not sure what the server side plans > around that work is. > > On Mon, Apr 4, 2016 at 3:18 AM Andrew Wilkins < > andrew.wilk...@canonical.com> wrote: > >> Hi, >> >> I would like to write a charm that should be mostly identical on Windows >> and Linux, so I think it would make sense to have common code in the form >> of a layer. >> >> Is anyone working on getting "charm build", layers, and friends to work >> with Windows workloads? If not, I may look into it myself. >> >> Cheers, >> Andrew >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju >> > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Action return value too long for commandline
Thanks, this is interesting to know. The fact that we've fixed it in relation set means we have a pattern to move forward with. The team's slammed on 2.0 work right now, but I've set this up to be something we try to address across the board after the fact. We'll make sure to look for other -set operations to make sure we standardize it across the board. On Sun, Apr 3, 2016 at 10:34 PM Stuart Bishop wrote: > On 4 April 2016 at 02:00, Merlijn Sebrechts > wrote: > > Hi all > > > > > > The apache-kafka charm has an action "read-topic" that can return a lot > of > > data. Sometimes, this data is too long to be passed to `action-set` by > > commandline. You get the following error: > > > > Traceback (most recent call last): > > File "actions/read-topic", line 36, in > > hookenv.action_set({'output': output}) > > File > > "/usr/local/lib/python2.7/dist-packages/charmhelpers/core/hookenv.py", > line > > 615, in action_set > > subprocess.check_call(cmd) > > File "/usr/lib/python2.7/subprocess.py", line 535, in check_call > > retcode = call(*popenargs, **kwargs) > > File "/usr/lib/python2.7/subprocess.py", line 522, in call > > return Popen(*popenargs, **kwargs).wait() > > File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ > > errread, errwrite) > > File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child > > raise child_exception > > OSError: [Errno 7] Argument list too long > > > > > > Is there any way to pass a file to action-set? > > This bug affects many of the Juju tools causing charms to fail at > scale. https://bugs.launchpad.net/juju-core/+bug/1437366 > (relation-set) is the only one I know of that has been fixed. > https://bugs.launchpad.net/juju-core/+bug/1274460 (juju-log) is still > open. leader-set also fails now I think of it, but I haven't tripped > over that one (it limits scalability the way I'm using leadershp > settings, but I should be able to squeeze out several hundred units). > Maybe we can use the opportunity to fix quoting and encoding issues. > > I suspect if you can get past command line length limitations, I > suspect the next glass ceiling is a 16MB document size limit in > MongoDB (which is large enough to not need fixing?) > > -- > Stuart Bishop > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Charm layers & Windows
I know that Gabriel and some of the CloudBase folks seemed interested in layers and possibly some tooling with powershell. I'm not sure how far that went but I thought they were experimenting during the charmer's summit. That would help with a charm build on windows, but not for some common code between both operating systems. An interesting thing is how much setup and how ootb the Ubuntu on Windows needs. If it's working out of the box, it might be an interesting move for us and our tools that Windows users could get a Linux experience. I guess that it won't be ideal though as I'm not sure what the server side plans around that work is. On Mon, Apr 4, 2016 at 3:18 AM Andrew Wilkins wrote: > Hi, > > I would like to write a charm that should be mostly identical on Windows > and Linux, so I think it would make sense to have common code in the form > of a layer. > > Is anyone working on getting "charm build", layers, and friends to work > with Windows workloads? If not, I may look into it myself. > > Cheers, > Andrew > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Charm layers & Windows
Hi, I would like to write a charm that should be mostly identical on Windows and Linux, so I think it would make sense to have common code in the form of a layer. Is anyone working on getting "charm build", layers, and friends to work with Windows workloads? If not, I may look into it myself. Cheers, Andrew -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju