[GitHub] bigtop pull request #383: BIGTOP-3065: Bump Hadoop to 2.8.4
GitHub user JunHe77 opened a pull request: https://github.com/apache/bigtop/pull/383 BIGTOP-3065: Bump Hadoop to 2.8.4 Bump to latest release on 2.8 branch Change-Id: I58e719ef66174312fd005b0bb0d11711aee69ad0 Signed-off-by: Jun He You can merge this pull request into a Git repository by running: $ git pull https://github.com/JunHe77/bigtop branch-1.3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/383.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #383 commit 4fab5869449d34ccddfaf7e8aa45244bdff6af97 Author: Jun He Date: 2018-08-15T07:07:42Z BIGTOP-3065: Bump Hadoop to 2.8.4 Bump to latest release on 2.8 branch Change-Id: I58e719ef66174312fd005b0bb0d11711aee69ad0 Signed-off-by: Jun He ---
[jira] [Created] (BIGTOP-3072) Ambari build fails
Valery Vybornov created BIGTOP-3072: --- Summary: Ambari build fails Key: BIGTOP-3072 URL: https://issues.apache.org/jira/browse/BIGTOP-3072 Project: Bigtop Issue Type: Bug Components: build Affects Versions: 1.3.0 Environment: CentOS 7.4 slave:trunk-centos-7 docker image Reporter: Valery Vybornov Attachments: bigtop-1.3-ambari-2.6.1-jline.patch Ambari (version 2.6.1.0.0) build fails with the following errors {code:java} [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /ws/build/ambari/rpm/BUILD/apache-ambari-2.6.1-src/ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java:[74,22] package jline.internal does not exist [ERROR] /ws/build/ambari/rpm/BUILD/apache-ambari-2.6.1-src/ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java:[372,7] cannot find symbol symbol: variable Log location: class org.apache.ambari.server.state.repository.VersionDefinitionXml [INFO] 2 errors {code} This seems to have been fixed here: [https://github.com/apache/ambari/pull/496] (in ambari [release-2.7.0-rc0|https://github.com/apache/ambari/releases/tag/release-2.7.0-rc0]); the [commit|https://github.com/apache/ambari/commit/2c58ee20d02e9695c52d8c5b347c5cccfb5cfcc0] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] bigtop pull request #382: Bigtop-3070 support Spark Thrift Server in Charm
Github user kwmonroe commented on a diff in the pull request: https://github.com/apache/bigtop/pull/382#discussion_r212387730 --- Diff: bigtop-packages/src/charm/spark/layer-spark/reactive/spark.py --- @@ -236,7 +235,8 @@ def reconfigure_spark(): # Almost all config changes should trigger a reinstall... except when # changing the bigtop repo version. Repo version changes require the user # to run an action, so we skip the reinstall in that case. -if not is_state('config.changed.bigtop_version'): +if not is_state('config.changed.bigtop_version') \ +and not is_state('config.changed.spark_enable_thriftserver'): --- End diff -- I'm a bit unsettled with this. If someone changed multiple config options with something like `juju config spark driver_memory=2048M spark_enable_thriftserver=true`, we'd miss the driver_mem change. What if we adjusted the `reconfigure_spark` function to be: ``` @when('spark.started', 'config.changed') @when_not('config.changed.bigtop_version', 'config.changed.spark_enable_thriftserver') def reconfigure_spark(): ... ``` We could then remove the check for `config.changed.bigtop_version` in the `reconfigure_spark` handler (it gets handled anyway with the `check_repo_version` function a few lines down). Then we could adjust your `start_thrift` function to be: ``` @when('spark.started', 'config.changed.spark_enable_thriftserver') def start_thrift(): enable_thrift = hookenv.config()['spark_enable_thriftserver'] if enable_thrift: check_call(['/usr/lib/spark/sbin/start-thriftserver.sh']) else: check_call(['/usr/lib/spark/sbin/stop-thriftserver.sh']) ``` ---
[GitHub] bigtop issue #382: Bigtop-3070 support Spark Thrift Server in Charm
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/382 One other thing @buggtb, since this opens a new port, we'll need 1 listed in layer.yaml: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/spark/layer-spark/layer.yaml#L25 As well as a check for `spark_enable_thriftserver` in the `start` method to know if we should run `hookenv.open_port`, similar to what we do for opening the spark-master-ui port: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/spark/layer-spark/lib/charms/layer/bigtop_spark.py#L377 ---