Re: Sending binaries over relations
It does feel like a good fit for resources, with the one caveat that he wants to maintain a lock-step version of the resource across services. There is slightly more work with the current designs for resources, in that each charm will think about its version of the resource independently. But we will have the fingerprint information to allow for users to compare and be confident that both services are using the same resource. John =:-> On Wed, Jan 20, 2016 at 8:53 PM, Mark Shuttleworth wrote: > On 20/01/16 14:24, Merlijn Sebrechts wrote: > > So my question is: Is there a way to send large binary files between > > Charms? Or is this problem better solved by using a subordinate > > kafka-plugin Charm like the Hadoop Charms do? > > It sounds like you want the new "Resources" capability coming in Juju 2.0 > :) > > For shared large blobs (like a JVM or a big ball of libraries) the charm > can ask the state server to cache the blob and distribute it to all the > units. There are mechanisms for users to supply the blob if needed, too. > > Mark > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Juju stable 1.25.3 is proposed for release
# juju-core 1.25.3 A new proposed stable release of Juju, juju-core 1.25.3, is now available. This release may replace version 1.25.0 on Thursday January 21. ## Getting Juju juju-core 1.25.3 is available for Xenial and backported to earlier series in the following PPA: https://launchpad.net/~juju/+archive/proposed Windows, Centos, and OS X users will find installers at: https://launchpad.net/juju-core/+milestone/1.25.3 Proposed releases use the "proposed" simple-streams. You must configure the `agent-stream` option in your environments.yaml to use the matching juju agents. ## Notable Changes This releases addresses a defect discovered in proposed 1.25.2 ## Resolved issues * Unit loses network connectivity during bootstrap: juju 1.25.2 + maas 1.9 Lp 1534795 * "cannot allocate memory" when running "juju run" Lp 1382556 * Bootstrap with the vsphere provider fails to log into the virtual machine Lp 1511138 * Add-machine with vsphere triggers machine-0: panic: juju home hasn't been initialized Lp 1513492 * Using maas 1.9 as provider using dhcp nic will prevent juju bootstrap Lp 1512371 * Worker/storageprovisioner: machine agents attempting to attach environ-scoped volumes Lp 1483492 * Restore: agent old password not found in configuration Lp 1452082 * "ignore-machine-addresses" broken for containers Lp 1509292 * Deploying a service to a space which has no subnets causes the agent to panic Lp 1499426 * /var/lib/juju gone after 1.18->1.20 upgrade and manual edit of agent.conf Lp 1444912 * Juju bootstrap fails to successfully configure the bridge juju-br0 when deploying with wily 4.2 kernel Lp 1496972 * Incompatible cookie format change Lp 1511717 * Error environment destruction failed: destroying storage: listing volumes: get https://x.x.x.x:8776/v2//volumes/detail: local error: record overflow Lp 1512399 * Replica set emptyconfig maas bootstrap Lp 1412621 * Juju can't find daily image streams from cloud- images.ubuntu.com/daily Lp 1513982 * Rsyslog certificate fails when using ipv6/4 dual stack with prefer-ipv6: true Lp 1478943 * Improper address:port joining Lp 1518128 * Juju status broken Lp 1516989 * 1.25.1 with maas 1.8: devices dns allocation uses non-unique hostname Lp 1525280 * Increment minimum juju version for 2.0 upgrade to 1.25.3 Lp 1533751 * Make assignment of units to machines use a worker Lp 1497312 * `juju environments` fails due to missing ~/.juju/current- environment Lp 1506680 * Juju 1.25 misconfigures juju-br0 when using maas 1.9 bonded interface Lp 1516891 * Destroy-environment on an unbootstrapped maas environment can release all my nodes Lp 1490865 * On juju upgrade the security group lost ports for the exposed services Lp 1506649 * Support centos and windows image metadata Lp 1523693 * Upgrade-juju shows available tools and best version but did not output what it decided to do Lp 1403655 * Invalid binary version, version "1.23.3--amd64" or "1.23.3--armhf" Lp 1459033 * Add xenial to supported series Lp 1533262 Finally We encourage everyone to subscribe the mailing list at juju-...@lists.canonical.com, or join us on #juju-dev on freenode. -- Curtis Hovey Canonical Cloud Development and Operations http://launchpad.net/~sinzui -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Sending binaries over relations
On 20 January 2016 at 22:16, Marco Ceppi wrote: > I don't think sending the binary via relation is a good idea. Either > spinning up a web service or using rsync would be a better bet Yeah. And you can't set relation data to anything larger than the maximum command line length, so it would only work as transport for small files. I'd go ssh myself, since most of it is already setup. charmhelpers.contrib.unison had code to setup the keys between peers (I've used this bit and it works, but not the actual unison bit). -- Stuart Bishop -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Sending binaries over relations
On 20/01/16 14:24, Merlijn Sebrechts wrote: > So my question is: Is there a way to send large binary files between > Charms? Or is this problem better solved by using a subordinate > kafka-plugin Charm like the Hadoop Charms do? It sounds like you want the new "Resources" capability coming in Juju 2.0 :) For shared large blobs (like a JVM or a big ball of libraries) the charm can ask the state server to cache the blob and distribute it to all the units. There are mechanisms for users to supply the blob if needed, too. Mark -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Sending binaries over relations
I don't think sending the binary via relation is a good idea. Either spinning up a web service or using rsync would be a better bet On Wed, Jan 20, 2016, 10:10 AM Matthew Williams < matthew.willi...@canonical.com> wrote: > Would it not be better for the charm to have a path the client can `wget` > the libraries from - this path can be sent via the relation as a string > > Matty > > On Wed, Jan 20, 2016 at 2:30 PM, José Antonio Rey wrote: > >> Hey, >> >> One of the options would be to cat the file as a string and pass that >> string over the connection, finally echoing that string to foo.binary. >> >> What do others think? >> >> -- >> José Antonio Rey >> >> On Wed, Jan 20, 2016, 08:25 Merlijn Sebrechts < >> merlijn.sebrec...@gmail.com> wrote: >> >>> Hi >>> >>> >>> I have a question I'd like to discuss, if you guys aren't to busy >>> prepping for Ubucon.. :) >>> >>> I've found a number of Java projects where, in order to communicate for >>> example with Kafka, they require the Kafka Java libraries for that specific >>> version. For the moment, I solve this by downloading the libraries from a >>> deployed Kafka installation and include them in the Charm. However, this >>> has the disadvantage that everytime the Kafka charm version changes, I have >>> to update the libraries in all the charms that connect to Kafka. It would >>> be better if there was a way to send these libraries over the connection. >>> This way, a Charm that can connect to one version of Kafka has a very high >>> chance of being able to connect to the next version. >>> >>> So my question is: Is there a way to send large binary files between >>> Charms? Or is this problem better solved by using a subordinate >>> kafka-plugin Charm like the Hadoop Charms do? >>> >>> >>> >>> Kind regards >>> Merlijn Sebrechts >>> -- >>> Juju mailing list >>> Juju@lists.ubuntu.com >>> Modify settings or unsubscribe at: >>> https://lists.ubuntu.com/mailman/listinfo/juju >>> >> >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju >> >> > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Sending binaries over relations
It seems rssh might be able to do this! 2016-01-20 16:25 GMT+01:00 Merlijn Sebrechts : > I like the idea of rsync. Is there a way to restrict access to a single > file on the rsync server? > > 2016-01-20 16:16 GMT+01:00 Marco Ceppi : > >> I don't think sending the binary via relation is a good idea. Either >> spinning up a web service or using rsync would be a better bet >> >> On Wed, Jan 20, 2016, 10:10 AM Matthew Williams < >> matthew.willi...@canonical.com> wrote: >> >>> Would it not be better for the charm to have a path the client can >>> `wget` the libraries from - this path can be sent via the relation as a >>> string >>> >>> Matty >>> >>> On Wed, Jan 20, 2016 at 2:30 PM, José Antonio Rey >>> wrote: >>> Hey, One of the options would be to cat the file as a string and pass that string over the connection, finally echoing that string to foo.binary. What do others think? -- José Antonio Rey On Wed, Jan 20, 2016, 08:25 Merlijn Sebrechts < merlijn.sebrec...@gmail.com> wrote: > Hi > > > I have a question I'd like to discuss, if you guys aren't to busy > prepping for Ubucon.. :) > > I've found a number of Java projects where, in order to communicate > for example with Kafka, they require the Kafka Java libraries for that > specific version. For the moment, I solve this by downloading the > libraries > from a deployed Kafka installation and include them in the Charm. However, > this has the disadvantage that everytime the Kafka charm version changes, > I > have to update the libraries in all the charms that connect to Kafka. It > would be better if there was a way to send these libraries over the > connection. This way, a Charm that can connect to one version of Kafka has > a very high chance of being able to connect to the next version. > > So my question is: Is there a way to send large binary files between > Charms? Or is this problem better solved by using a subordinate > kafka-plugin Charm like the Hadoop Charms do? > > > > Kind regards > Merlijn Sebrechts > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju >>> -- >>> Juju mailing list >>> Juju@lists.ubuntu.com >>> Modify settings or unsubscribe at: >>> https://lists.ubuntu.com/mailman/listinfo/juju >>> >> >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju >> >> > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Sending binaries over relations
I like the idea of rsync. Is there a way to restrict access to a single file on the rsync server? 2016-01-20 16:16 GMT+01:00 Marco Ceppi : > I don't think sending the binary via relation is a good idea. Either > spinning up a web service or using rsync would be a better bet > > On Wed, Jan 20, 2016, 10:10 AM Matthew Williams < > matthew.willi...@canonical.com> wrote: > >> Would it not be better for the charm to have a path the client can `wget` >> the libraries from - this path can be sent via the relation as a string >> >> Matty >> >> On Wed, Jan 20, 2016 at 2:30 PM, José Antonio Rey >> wrote: >> >>> Hey, >>> >>> One of the options would be to cat the file as a string and pass that >>> string over the connection, finally echoing that string to foo.binary. >>> >>> What do others think? >>> >>> -- >>> José Antonio Rey >>> >>> On Wed, Jan 20, 2016, 08:25 Merlijn Sebrechts < >>> merlijn.sebrec...@gmail.com> wrote: >>> Hi I have a question I'd like to discuss, if you guys aren't to busy prepping for Ubucon.. :) I've found a number of Java projects where, in order to communicate for example with Kafka, they require the Kafka Java libraries for that specific version. For the moment, I solve this by downloading the libraries from a deployed Kafka installation and include them in the Charm. However, this has the disadvantage that everytime the Kafka charm version changes, I have to update the libraries in all the charms that connect to Kafka. It would be better if there was a way to send these libraries over the connection. This way, a Charm that can connect to one version of Kafka has a very high chance of being able to connect to the next version. So my question is: Is there a way to send large binary files between Charms? Or is this problem better solved by using a subordinate kafka-plugin Charm like the Hadoop Charms do? Kind regards Merlijn Sebrechts -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju >>> >>> -- >>> Juju mailing list >>> Juju@lists.ubuntu.com >>> Modify settings or unsubscribe at: >>> https://lists.ubuntu.com/mailman/listinfo/juju >>> >>> >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju >> > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Sending binaries over relations
@José: I worry that this will become a big strain to the state server. I don't know much about the internal architecture to be certain of this, though.. @Matthew: This would side-step jujuresources. It might be feasible if jujuresources could provide a way for the Charm to Share that resource, though... Even then, we might get into trouble if the binaries aren't included as a resource but are compiled during installation... 2016-01-20 16:10 GMT+01:00 Matthew Williams : > Would it not be better for the charm to have a path the client can `wget` > the libraries from - this path can be sent via the relation as a string > > Matty > > On Wed, Jan 20, 2016 at 2:30 PM, José Antonio Rey wrote: > >> Hey, >> >> One of the options would be to cat the file as a string and pass that >> string over the connection, finally echoing that string to foo.binary. >> >> What do others think? >> >> -- >> José Antonio Rey >> >> On Wed, Jan 20, 2016, 08:25 Merlijn Sebrechts < >> merlijn.sebrec...@gmail.com> wrote: >> >>> Hi >>> >>> >>> I have a question I'd like to discuss, if you guys aren't to busy >>> prepping for Ubucon.. :) >>> >>> I've found a number of Java projects where, in order to communicate for >>> example with Kafka, they require the Kafka Java libraries for that specific >>> version. For the moment, I solve this by downloading the libraries from a >>> deployed Kafka installation and include them in the Charm. However, this >>> has the disadvantage that everytime the Kafka charm version changes, I have >>> to update the libraries in all the charms that connect to Kafka. It would >>> be better if there was a way to send these libraries over the connection. >>> This way, a Charm that can connect to one version of Kafka has a very high >>> chance of being able to connect to the next version. >>> >>> So my question is: Is there a way to send large binary files between >>> Charms? Or is this problem better solved by using a subordinate >>> kafka-plugin Charm like the Hadoop Charms do? >>> >>> >>> >>> Kind regards >>> Merlijn Sebrechts >>> -- >>> Juju mailing list >>> Juju@lists.ubuntu.com >>> Modify settings or unsubscribe at: >>> https://lists.ubuntu.com/mailman/listinfo/juju >>> >> >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju >> >> > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Sending binaries over relations
Would it not be better for the charm to have a path the client can `wget` the libraries from - this path can be sent via the relation as a string Matty On Wed, Jan 20, 2016 at 2:30 PM, José Antonio Rey wrote: > Hey, > > One of the options would be to cat the file as a string and pass that > string over the connection, finally echoing that string to foo.binary. > > What do others think? > > -- > José Antonio Rey > > On Wed, Jan 20, 2016, 08:25 Merlijn Sebrechts > wrote: > >> Hi >> >> >> I have a question I'd like to discuss, if you guys aren't to busy >> prepping for Ubucon.. :) >> >> I've found a number of Java projects where, in order to communicate for >> example with Kafka, they require the Kafka Java libraries for that specific >> version. For the moment, I solve this by downloading the libraries from a >> deployed Kafka installation and include them in the Charm. However, this >> has the disadvantage that everytime the Kafka charm version changes, I have >> to update the libraries in all the charms that connect to Kafka. It would >> be better if there was a way to send these libraries over the connection. >> This way, a Charm that can connect to one version of Kafka has a very high >> chance of being able to connect to the next version. >> >> So my question is: Is there a way to send large binary files between >> Charms? Or is this problem better solved by using a subordinate >> kafka-plugin Charm like the Hadoop Charms do? >> >> >> >> Kind regards >> Merlijn Sebrechts >> -- >> Juju mailing list >> Juju@lists.ubuntu.com >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju >> > > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Re: Sending binaries over relations
Hey, One of the options would be to cat the file as a string and pass that string over the connection, finally echoing that string to foo.binary. What do others think? -- José Antonio Rey On Wed, Jan 20, 2016, 08:25 Merlijn Sebrechts wrote: > Hi > > > I have a question I'd like to discuss, if you guys aren't to busy prepping > for Ubucon.. :) > > I've found a number of Java projects where, in order to communicate for > example with Kafka, they require the Kafka Java libraries for that specific > version. For the moment, I solve this by downloading the libraries from a > deployed Kafka installation and include them in the Charm. However, this > has the disadvantage that everytime the Kafka charm version changes, I have > to update the libraries in all the charms that connect to Kafka. It would > be better if there was a way to send these libraries over the connection. > This way, a Charm that can connect to one version of Kafka has a very high > chance of being able to connect to the next version. > > So my question is: Is there a way to send large binary files between > Charms? Or is this problem better solved by using a subordinate > kafka-plugin Charm like the Hadoop Charms do? > > > > Kind regards > Merlijn Sebrechts > -- > Juju mailing list > Juju@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju > -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
Sending binaries over relations
Hi I have a question I'd like to discuss, if you guys aren't to busy prepping for Ubucon.. :) I've found a number of Java projects where, in order to communicate for example with Kafka, they require the Kafka Java libraries for that specific version. For the moment, I solve this by downloading the libraries from a deployed Kafka installation and include them in the Charm. However, this has the disadvantage that everytime the Kafka charm version changes, I have to update the libraries in all the charms that connect to Kafka. It would be better if there was a way to send these libraries over the connection. This way, a Charm that can connect to one version of Kafka has a very high chance of being able to connect to the next version. So my question is: Is there a way to send large binary files between Charms? Or is this problem better solved by using a subordinate kafka-plugin Charm like the Hadoop Charms do? Kind regards Merlijn Sebrechts -- Juju mailing list Juju@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju
