[GitHub] bigtop pull request #383: BIGTOP-3065: Bump Hadoop to 2.8.4
GitHub user JunHe77 opened a pull request: https://github.com/apache/bigtop/pull/383 BIGTOP-3065: Bump Hadoop to 2.8.4 Bump to latest release on 2.8 branch Change-Id: I58e719ef66174312fd005b0bb0d11711aee69ad0 Signed-off-by: Jun He You can merge this pull request into a Git repository by running: $ git pull https://github.com/JunHe77/bigtop branch-1.3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/bigtop/pull/383.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #383 commit 4fab5869449d34ccddfaf7e8aa45244bdff6af97 Author: Jun He Date: 2018-08-15T07:07:42Z BIGTOP-3065: Bump Hadoop to 2.8.4 Bump to latest release on 2.8 branch Change-Id: I58e719ef66174312fd005b0bb0d11711aee69ad0 Signed-off-by: Jun He ---
[jira] [Created] (BIGTOP-3072) Ambari build fails
Valery Vybornov created BIGTOP-3072:
---
Summary: Ambari build fails
Key: BIGTOP-3072
URL: https://issues.apache.org/jira/browse/BIGTOP-3072
Project: Bigtop
Issue Type: Bug
Components: build
Affects Versions: 1.3.0
Environment: CentOS 7.4
slave:trunk-centos-7 docker image
Reporter: Valery Vybornov
Attachments: bigtop-1.3-ambari-2.6.1-jline.patch
Ambari (version 2.6.1.0.0) build fails with the following errors
{code:java}
[INFO] -
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR]
/ws/build/ambari/rpm/BUILD/apache-ambari-2.6.1-src/ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java:[74,22]
package jline.internal does not exist
[ERROR]
/ws/build/ambari/rpm/BUILD/apache-ambari-2.6.1-src/ambari-server/src/main/java/org/apache/ambari/server/state/repository/VersionDefinitionXml.java:[372,7]
cannot find symbol
symbol: variable Log
location: class org.apache.ambari.server.state.repository.VersionDefinitionXml
[INFO] 2 errors
{code}
This seems to have been fixed here: [https://github.com/apache/ambari/pull/496]
(in ambari
[release-2.7.0-rc0|https://github.com/apache/ambari/releases/tag/release-2.7.0-rc0]);
the
[commit|https://github.com/apache/ambari/commit/2c58ee20d02e9695c52d8c5b347c5cccfb5cfcc0]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] bigtop pull request #382: Bigtop-3070 support Spark Thrift Server in Charm
Github user kwmonroe commented on a diff in the pull request:
https://github.com/apache/bigtop/pull/382#discussion_r212387730
--- Diff: bigtop-packages/src/charm/spark/layer-spark/reactive/spark.py ---
@@ -236,7 +235,8 @@ def reconfigure_spark():
# Almost all config changes should trigger a reinstall... except when
# changing the bigtop repo version. Repo version changes require the
user
# to run an action, so we skip the reinstall in that case.
-if not is_state('config.changed.bigtop_version'):
+if not is_state('config.changed.bigtop_version') \
+and not is_state('config.changed.spark_enable_thriftserver'):
--- End diff --
I'm a bit unsettled with this. If someone changed multiple config options
with something like `juju config spark driver_memory=2048M
spark_enable_thriftserver=true`, we'd miss the driver_mem change.
What if we adjusted the `reconfigure_spark` function to be:
```
@when('spark.started', 'config.changed')
@when_not('config.changed.bigtop_version',
'config.changed.spark_enable_thriftserver')
def reconfigure_spark():
...
```
We could then remove the check for `config.changed.bigtop_version` in the
`reconfigure_spark` handler (it gets handled anyway with the
`check_repo_version` function a few lines down). Then we could adjust your
`start_thrift` function to be:
```
@when('spark.started', 'config.changed.spark_enable_thriftserver')
def start_thrift():
enable_thrift = hookenv.config()['spark_enable_thriftserver']
if enable_thrift:
check_call(['/usr/lib/spark/sbin/start-thriftserver.sh'])
else:
check_call(['/usr/lib/spark/sbin/stop-thriftserver.sh'])
```
---
[GitHub] bigtop issue #382: Bigtop-3070 support Spark Thrift Server in Charm
Github user kwmonroe commented on the issue: https://github.com/apache/bigtop/pull/382 One other thing @buggtb, since this opens a new port, we'll need 1 listed in layer.yaml: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/spark/layer-spark/layer.yaml#L25 As well as a check for `spark_enable_thriftserver` in the `start` method to know if we should run `hookenv.open_port`, similar to what we do for opening the spark-master-ui port: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/charm/spark/layer-spark/lib/charms/layer/bigtop_spark.py#L377 ---
