Re: [ANN] Bigtop-1.2 charms/bundles have been released

2017-04-26 Thread Mark Shuttleworth 
On 26/04/17 03:37, Merlijn Sebrechts wrote:
> Awesome!
>
> Some visibility improvements.
>
>   * If you search for Hadoop  in the
> Charm Store, the first results you get are the old non-bigtop
> charms. Can we do something about this? This is the same for
> Spark, Pig, etc..
>   * The ancient "hadoop" 
> Charms are still in the store and stil "recommended".
>   * If you search for "bigtop", you only see one bundle while there
> are a bunch of bigtop
> bundles: https://jujucharms.com/q/bigtop?type=bundle
> 
>   * If you click on the search bar in the store without typing
> something[1] we see a bunch of old bigdata charms. This should
> show the latest and gratest.
>   * The bigtop Spark charm doesn't seem to have bigtop in its
> icon? https://jujucharms.com/spark/ 
>   * There are still a bunch of charms that don't list what version
> they install in the description.
>

Thanks Merlijn, those are good catches and, I trust, quick and easy fixes.

Mark
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

Re: [ANN] Bigtop-1.2 charms/bundles have been released

2017-04-26 Thread Merlijn Sebrechts 
Awesome!

Some visibility improvements.

   - If you search for Hadoop  in the
   Charm Store, the first results you get are the old non-bigtop charms. Can
   we do something about this? This is the same for Spark, Pig, etc..
   - The ancient "hadoop"  Charms
   are still in the store and stil "recommended".
   - If you search for "bigtop", you only see one bundle while there are a
   bunch of bigtop bundles: https://jujucharms.com/q/bigtop?type=bundle
   - If you click on the search bar in the store without typing
   something[1] we see a bunch of old bigdata charms. This should show the
   latest and gratest.
   - The bigtop Spark charm doesn't seem to have bigtop in its icon?
   https://jujucharms.com/spark/ 
   - There are still a bunch of charms that don't list what version they
   install in the description.


[1]
[image: Inline afbeelding 1]

2017-04-25 18:54 GMT+02:00 Kevin Monroe :

> Hi folks!
>
> After over a year in development, Apache Bigtop 1.2 was released earlier
> this month.  Today, we released updated charms and bundles for this release
> to the appropriate stable channels.
>
> HIGHLIGHTS:
>
> + Software versions:
> - hadoop-namenode 2.7.3
> - hadoop-plugin 2.7.3
> - hadoop-resourcemanager 2.7.3
> - hadoop-slave 2.7.3
> - hbase 1.1.3-1 **
> - kafka 0.10.1.1-1
> - mahout 0.12.2-1 **
> - pig 0.15.0
> - spark 2.1.0-1
> - zeppelin 0.7.0
> - zookeeper 3.4.6-1 **
>
> ** There is a planned bigtop-1.2.1 point release to bring in hotfixes.
> This will likely include hbase-1.1.9, mahout-0.13, and zookeeper-3.4.10.
> No charm changes are required; when these debs hit the upstream repo, the
> charms will provide the new version.
>
> + Juju 2.0 or greater
> - To leverage Juju Resources, bigtop-related charms now require Juju 2 or
> greater.
>
> + Usability
> - Refreshed READMEs and actions for a consistent UX
>
> + Hadoop-Spark
> - Reduced resource requirements.  Spark is not in HA mode in this bundle,
> so we removed 3 unnecessary zookeeper units.
>
> + Spark
> - New configuration to change driver/executor memory at runtime.
> - Improved spark reliability when changing execution mode (local,
> standalone, yarn).
>
>
> KNOWN ISSUES:
>
> + GCE
> - https://bugs.launchpad.net/juju/+bug/1674871
> - Most of our applications require a minimum of 7g ram.  We set a generic
> memory constraint in our bundles for a consistent experience across all
> clouds.  However, with GCE, mem=7G results in a "highcpu" instance type.
> These are significantly more expensive than "standard" instance types.
> - If "highcpu" instances are not required, workaround this issue by
> modifying a local copy of bundle.yaml.  For example:
>
> $ charm pull hadoop-processing
> cs:bundle/hadoop-processing-58
> $ cd hadoop-processing/
> $ sed -ie 's/mem=7G/instance-type=n1-standard-2/' bundle.yaml
> $ juju deploy ./bundle.yaml
>
> + Zeppelin
> - https://issues.apache.org/jira/browse/BIGTOP-2742
> - The init script for the zeppelin service is broken.  The charm will work
> around this during install and relation changes, but users may find that
> things like "sudo systemctl  zeppelin" on the unit do not work.
> - We'll update the zeppelin charm once an appropriate fix lands upstream.
>
>
> Please let me know if you have any questions or issues with the bigtop-1.2
> release.  Thanks!
> -Kevin Monroe
>
> --
> Juju mailing list
> Juju@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/juju
>
>
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju

[ANN] Bigtop-1.2 charms/bundles have been released

2017-04-25 Thread Kevin Monroe 
Hi folks!

After over a year in development, Apache Bigtop 1.2 was released earlier
this month.  Today, we released updated charms and bundles for this release
to the appropriate stable channels.

HIGHLIGHTS:

+ Software versions:
- hadoop-namenode 2.7.3
- hadoop-plugin 2.7.3
- hadoop-resourcemanager 2.7.3
- hadoop-slave 2.7.3
- hbase 1.1.3-1 **
- kafka 0.10.1.1-1
- mahout 0.12.2-1 **
- pig 0.15.0
- spark 2.1.0-1
- zeppelin 0.7.0
- zookeeper 3.4.6-1 **

** There is a planned bigtop-1.2.1 point release to bring in hotfixes.
This will likely include hbase-1.1.9, mahout-0.13, and zookeeper-3.4.10.
No charm changes are required; when these debs hit the upstream repo, the
charms will provide the new version.

+ Juju 2.0 or greater
- To leverage Juju Resources, bigtop-related charms now require Juju 2 or
greater.

+ Usability
- Refreshed READMEs and actions for a consistent UX

+ Hadoop-Spark
- Reduced resource requirements.  Spark is not in HA mode in this bundle,
so we removed 3 unnecessary zookeeper units.

+ Spark
- New configuration to change driver/executor memory at runtime.
- Improved spark reliability when changing execution mode (local,
standalone, yarn).


KNOWN ISSUES:

+ GCE
- https://bugs.launchpad.net/juju/+bug/1674871
- Most of our applications require a minimum of 7g ram.  We set a generic
memory constraint in our bundles for a consistent experience across all
clouds.  However, with GCE, mem=7G results in a "highcpu" instance type.
These are significantly more expensive than "standard" instance types.
- If "highcpu" instances are not required, workaround this issue by
modifying a local copy of bundle.yaml.  For example:

$ charm pull hadoop-processing
cs:bundle/hadoop-processing-58
$ cd hadoop-processing/
$ sed -ie 's/mem=7G/instance-type=n1-standard-2/' bundle.yaml
$ juju deploy ./bundle.yaml

+ Zeppelin
- https://issues.apache.org/jira/browse/BIGTOP-2742
- The init script for the zeppelin service is broken.  The charm will work
around this during install and relation changes, but users may find that
things like "sudo systemctl  zeppelin" on the unit do not work.
- We'll update the zeppelin charm once an appropriate fix lands upstream.


Please let me know if you have any questions or issues with the bigtop-1.2
release.  Thanks!
-Kevin Monroe
-- 
Juju mailing list
Juju@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/juju