Re: Ambati GUI doesn’t start

[David Quiroga]

If Ambari Server reported successful start up and logs seem to confirm that
but you are unable to access the UI with a browser

Would suggest checking "firewall" settings the port may be blocked from
external access.

On Sat, Jul 7, 2018 at 10:30 PM, Shekhar Babulal Dhotre <
shekhar.dho...@biascorp.com> wrote:

> Hello folks ,
> I have installed Anandi on Centos hosted on google cloud . All install
> went as expected and and amabari sever also starts . When I go to browser
> with iP and port it just hangs in their .
> I
> Will send logs to you .
> What other settings should I check ?
>
> Thanks
> SD
>
> Get Outlook for iOS 
>

Re: Modifying a Stack with an Extension

[David Quiroga]

We have been able to add the management of HBase Thrift and Rest into our
Ambari via custom management packs.

https://cwiki.apache.org/confluence/display/AMBARI/Management+Packs#ManagementPacks-Add%C2%AD-OnServiceManagementPackStructure

In our mpacks there is a metainfo.xml under the common-services,
custom-services, and extensions folders.

I suspect the extensions might not work as expected because the services
don't exist in core HDP.



On Mon, Jun 25, 2018 at 10:53 AM, jeremy montgomery  wrote:

> So the following has to do with adding the HBase Thrift and Rest Server to
> Ambari.  Personally, I like using extensions because its isolates the
> functions from the upgrade process.  However, it doesn't look like an
> extension can extend either a stack or a common-service.
>
> Stacks => HDP 2.6, BigInsights 4.2.5
>
> Goals:
> Be able to add Thrift/Rest with configs
> Add appropriate alerts
> Recompile app.js to have them appear in the summary panel
>
> QUESTION 1 => is it possible to extend this with an extension?  Like
> Extend HDP 2.3 instead of common-services?
>
> QUESTION 2 => Is it possible to trigger a recompile of app.js with a flag?
>
> <<
> First Try
> Extension Pack
> 
>   2.0
>   
> 
>   HBASE
>   common-services/HBASE/0.96.0.2.0
> ###thrift and rest components
> 
>   
> 
>
> Result
> Not able to add Thrift/Rest Server
>
> Second Try:
> Modifying HDP 2.3 Stack HBase metainfo.xml to add them:
>
> Result:
> able to add Thrift/Rest server with configs
> add appropriate alerts
> Only Rest shows up in the summary panel since REST came with BigInsights
> 4.0 (but was lost when it merged with HDP 2.3) after digging, this seems to
> be because you need a stack change or extension to recompile app.js to
> include the new information.
>
>
> There also appears to be an inheritance problem when stacks operate on the
> same common-service.
>
> BigInsights 4.0 adds the HBase Rest Server, but this isn't anywhere in the
> HDP Stack so it doesn't show up as a possibility.  However, if you add a
> reference to the HDP 2.3 metainfo.xml, it will pick up all of the
> BigInsights code.
>
>

Re: when agent sync service scripts

[David Quiroga]

It should be near real-time.

I wonder if the agent logs may provide some direction.

For simple testing the code can be updated from the agent cache itself, but
this would be overwritten at the next check in with the server.

On Sun, May 20, 2018 at 6:43 PM, <xiang@sky-data.cn> wrote:

> In my case, i define a stack and install a service.
>
> For start/stop, i need to do some test, so i edit service script under
> stack.
>
> Then restart server, not sync.
>
> Then restart agent, still not.
>
> Does it need time to sync or it is real-time?
>
> ------
> *From: *"David Quiroga" <quirogadf4w...@gmail.com>
> *To: *"user" <user@ambari.apache.org>
> *Sent: *Monday, May 21, 2018 1:20:31 AM
> *Subject: *Re: when agent sync service scripts
>
> The agent code cache should update when the agents check-in to the server.
> Believe check-in occurs after the server is restarted and/or after an
> agent is restarted.
>
> Is the python? code in the standard location ambari-server/resources/
> common-services/YOUR_SERVICE/VERSION/package/scripts/
> Or what type of code update are you attempting.
>
>
> On Sat, May 19, 2018 at 1:37 AM, <xiang@sky-data.cn> wrote:
>
>> I install a custom stack in VM, then i want to update some codes for a
>> installed service.
>>
>> But when i restart server, the codes in agent does not update.
>>
>> Then i restart agent, still failed.
>>
>> So when agent sync service scripts?
>>
>> Thanks
>>
>
>

Re: when agent sync service scripts

[David Quiroga]

The agent code cache should update when the agents check-in to the server.
Believe check-in occurs after the server is restarted and/or after an agent
is restarted.

Is the python? code in the standard location
ambari-server/resources/common-services/YOUR_SERVICE/VERSION/package/scripts/
Or what type of code update are you attempting.


On Sat, May 19, 2018 at 1:37 AM,  wrote:

> I install a custom stack in VM, then i want to update some codes for a
> installed service.
>
> But when i restart server, the codes in agent does not update.
>
> Then i restart agent, still failed.
>
> So when agent sync service scripts?
>
> Thanks
>

Re: Operating System matching redhat7 could not be found

[David Quiroga]

So you are still seeing an error?

Was there more to the past error message?

We always remove the OS repositories we are not using from the listing.

What action were you attempting when the original problem occurred?

Re: Operating System matching redhat7 could not be found

[David Quiroga]

What is the output of cat /etc/*release on the hosts.
I know the code has to map values to the 'redhat7'

On Fri, May 18, 2018 at 7:08 AM, <xiang@sky-data.cn> wrote:

> I define a stack called SKY, and my repo.xml is as below:
>
> 
> 
> 
>   http://public-repo-1.hortonworks.com/HDP/centos7/2.
> x/updates/2.6.0.3
>   SKY-3.0
>   SKY
>   true
> 
> 
>   http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.
> 0.21/repos/centos7
>   SKY-UTILS-1.1.0.21
>   SKY-UTILS
>   false
> 
>   
> 
>
> I use centos7 to test, i really do not know how to fix it.
>
> Could someone help me?
>
> Thanks
>
> --
> *From: *"David Quiroga" <quirogadf4w...@gmail.com>
> *To: *"user" <user@ambari.apache.org>
> *Sent: *Friday, May 18, 2018 1:29:13 AM
>
> *Subject: *Re: Operating System matching redhat7 could not be found
>
> *"At least one set of repositories for OS should be provided"*
>
> I would take a look in Ambari under Admin > Stacks and Versions > Versions
> Tab
> For the current version "Show Details", click Edit button this should list
> the OS and repositories.
>
> Are those valid from the host you are now running on?
>
> On Thu, May 17, 2018 at 7:38 AM, <xiang@sky-data.cn> wrote:
>
>> True error maybe as below:
>>
>> 17 May 2018 20:33:46,103 ERROR [ambari-client-thread-78]
>> AbstractResourceProvider:343 - Caught AmbariException when modifying a
>> resource
>> org.apache.ambari.server.AmbariException: At least one set of
>> repositories for OS should be provided
>> at org.apache.ambari.server.controller.internal.
>> RepositoryVersionResourceProvider.validateRepositoryVersion(
>> RepositoryVersionResourceProvider.java:490)
>> at org.apache.ambari.server.controller.internal.
>> RepositoryVersionResourceProvider.validateRepositoryVersion(
>> RepositoryVersionResourceProvider.java:439)
>> at org.apache.ambari.server.controller.internal.
>> RepositoryVersionResourceProvider$2.invoke(RepositoryVersionResourceProvi
>> der.java:356)
>> at org.apache.ambari.server.controller.internal.
>> RepositoryVersionResourceProvider$2.invoke(RepositoryVersionResourceProvi
>> der.java:316)
>> at org.apache.ambari.server.controller.internal.
>> AbstractResourceProvider.invokeWithRetry(AbstractResourceProvider.java:
>> 455)
>> at org.apache.ambari.server.controller.internal.
>> AbstractResourceProvider.modifyResources(AbstractResourceProvider.java:
>> 336)
>> at org.apache.ambari.server.controller.internal.
>> RepositoryVersionResourceProvider.updateResourcesAuthorized(
>> RepositoryVersionResourceProvider.java:316)
>> at org.apache.ambari.server.controller.internal.
>> RepositoryVersionResourceProvider$$EnhancerByGuice$$46398843.CGLIB$
>> updateResourcesAuthorized$2()
>> at org.apache.ambari.server.controller.internal.
>> RepositoryVersionResourceProvider$$EnhancerByGuice$$
>> 46398843$$FastClassByGuice$$bbfdd590.invoke()
>> at com.google.inject.internal.cglib.proxy.$MethodProxy.
>> invokeSuper(MethodProxy.java:228)
>> at com.google.inject.internal.InterceptorStackCallback$
>> InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>> at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.
>> invoke(AmbariJpaLocalTxnInterceptor.java:128)
>> at com.google.inject.internal.InterceptorStackCallback$
>> InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
>> at com.google.inject.internal.InterceptorStackCallback.intercept(
>> InterceptorStackCallback.java:52)
>> at org.apache.ambari.server.controller.internal.
>> RepositoryVersionResourceProvider$$EnhancerByGuice$$46398843.
>> updateResourcesAuthorized()
>> at org.apache.ambari.server.controller.internal.
>> AbstractAuthorizedResourceProvider.updateResources(
>> AbstractAuthorizedResourceProvider.java:301)
>> at org.apache.ambari.server.controller.internal.
>> ClusterControllerImpl.updateResources(ClusterControllerImpl.java:319)
>> at org.apache.ambari.server.api.services.persistence.
>> PersistenceManagerImpl.update(PersistenceManagerImpl.java:125)
>> at org.apache.ambari.server.api.handlers.UpdateHandler.
>> persist(UpdateHandler.java:45)
>>
>> --
>> *From: *"xiang dai" <xiang@sky-data.cn>
>> *To: *"user" <user@ambari

Re: make ambari create kerberos users in custom format

[David Quiroga]

The formatting of the principal name is just a property and while it could
be changed I believe the cluster name is usually added as Principal Suffix
because what if there are multiple clusters in the same Kerberos realm.

If the principal was only hdfs@domain then multiple clusters would share
the same KDC entry. Re-generating the keytab in one cluster might cause
issues in the others. Also potential security risk as it would allow cross
cluster access.

This kinit is correct
kinit  -kt /etc/security/keytabs/hdfs.headless.keytab hdfs-spark_cluster

Using principal: hdfs-spark_cluster@test_kdc.com

When you present that principal you will be treated like hdfs because of
the rule RULE:[1:$1@$0](hdfs-spark_cluster@test_kdc.com)s/.*/hdfs/


On Mon, May 7, 2018 at 6:14 PM, Lian Jiang  wrote:

> Hi,
>
> I am using HDP2.6 and have enabled kerberos. The rules generated by ambari
> has:
>
> RULE:[1:$1@$0](hdfs-spark_cluster@test_kdc.com)s/.*/hdfs/
>
> Also, klist shows hdfs user is mapped correctly to the rule:
>
> [hdfs@test-namenode ~]$ klist
> Ticket cache: KEYRING:persistent:1012:1012
> Default principal: hdfs-spark_cluster@test_kdc.com
>
> User hdfs-spark_cluster is associated with hdfs keytab:
>
> [hdfs@test-namenode ~]$ kinit -V -kt 
> /etc/security/keytabs/hdfs.headless.keytab
> hdfs-spark_cluster
> Using existing cache: persistent:1012:1012
> Using principal: hdfs-spark_cluster@test_kdc.com
> Using keytab: /etc/security/keytabs/hdfs.headless.keytab
> Authenticated to Kerberos v5
>
> However, hdfs is NOT associated with this hdfs keytab:
>
> [hdfs@test-namenode ~]$ kinit -V -kt 
> /etc/security/keytabs/hdfs.headless.keytab
> hdfs
> Using new cache: persistent:1012:krb_ccache_V36KQXp
> Using principal: hdfs@test_kdc.com
> Using keytab: /etc/security/keytabs/hdfs.headless.keytab
> kinit: Keytab contains no suitable keys for hdfs@test_kdc.com while
> getting initial credentials
>
> As you can see, kinit maps hdfs to hdfs@test_kdc.com instead of
> hdfs-spark_cluster@test_kdc.com.
>
> I guess this is the reason I got "Failed to find any Kerberos tgt" when
> doing "hdfs dfs -ls".
>
> I don't know why ambari create kerberos users in the format of
> "hdfs-{CLUSTERNAME}@{REALNAME}" instead of "hdfs@{REALNAME}".
>
> Should I follow https://community.hortonworks.com/articles/79574/build-a-
> cluster-with-custom-principal-names-using.html to force ambari to create
> hdfs@test_kdc.com instead of hdfs-spark_cluster@test_kdc.com? Or I am
> missing anything else?
>
> Thanks for any help.
>
>

Re: metrics can not work as expected

[David Quiroga]

In most cases Ambari Metrics runs separate HBase and Zookeeper instances.
The default ports of Ambari Metrics Zookeeper are typically in the 60,000
range.
Would expect the Ambari defaults to do the trick, which I suspect relates
to Sid's question.


On Sun, May 6, 2018 at 5:07 AM,  wrote:

> I just download them and make my own repo including them.
> Then, i install them with ambari UI.
>
>
>
> --
> *From: *"Siddharth Wagle" 
> *To: *"user" 
> *Sent: *Saturday, May 5, 2018 11:07:12 PM
> *Subject: *Re: metrics can not work as expected
>
> ​Collector is not able to reach Zookeeper. Are you not installing AMS
> using Ambari?
>
>
> - Sid
>
>
> --
> *From:* xiang@sky-data.cn 
> *Sent:* Saturday, May 5, 2018 1:27 AM
> *To:* user
> *Subject:* metrics can not work as expected
>
> Hi!
>
> I tested installation on my vbox, it works well, then i move it to a
> server which run many services.
>
> When install, it failed at check service status.
>
> I checked ambari-metrics-collector.log and found below error:
>
> 2018-05-05 08:16:03,536 INFO org.apache.phoenix.metrics.Metrics:
> Initializing metrics system: phoenix
> 2018-05-05 08:16:03,559 WARN org.apache.hadoop.metrics2.impl.MetricsConfig:
> Cannot locate configuration: tried hadoop-metrics2-phoenix.
> properties,hadoop-metrics2.properties
> 
>
> 2018-05-05 08:18:50,342 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to dx-1.novalocal/192.168.80.34:2188, initiating
> session
> 2018-05-05 08:18:50,343 INFO org.apache.zookeeper.ClientCnxn: Unable to
> read additional data from server sessionid 0x0, likely server has closed
> socket, closing socket connection and attempting reconnect
> 2018-05-05 08:18:50,811 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server dx-2.novalocal/192.168.80.27:2188. Will not
> attempt to authenticate using SASL (unknown error)
> 2018-05-05 08:18:50,812 INFO org.apache.zookeeper.ClientCnxn: Socket
> connection established to dx-2.novalocal/192.168.80.27:2188, initiating
> session
> 2018-05-05 08:18:50,813 INFO org.apache.zookeeper.ClientCnxn: Unable to
> read additional data from server sessionid 0x0, likely server has closed
> socket, closing socket connection and attempting reconnect
> 
>
> 2018-05-05 08:18:50,947 INFO org.apache.helix.manager.zk.ZkClient:
> Closing zkclient: State:CONNECTING sessionid:0x0 local:null
> remoteserver:null lastZxid:0 xid:1 sent:35 recv:0 queuedpkts:0
> pendingresp:0 queuedevents:0
> 2018-05-05 08:18:50,947 INFO org.I0Itec.zkclient.ZkEventThread: Terminate
> ZkClient event thread.
> 2018-05-05 08:18:51,352 INFO org.apache.zookeeper.ZooKeeper: Session: 0x0
> closed
> 2018-05-05 08:18:51,352 INFO org.apache.helix.manager.zk.ZkClient: Closed
> zkclient
> 2018-05-05 08:18:51,352 ERROR org.apache.hadoop.yarn.server.
> applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore:
> org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to
> zookeeper server within timeout: 3
> 2018-05-05 08:18:51,354 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.yarn.server.applicationhistoryservice.
> metrics.timeline.HBaseTimelineMetricStore failed in state INITED; cause:
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.
> MetricsSystemInitializationException: Unable to initialize HA controller
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.
> MetricsSystemInitializationException: Unable to initialize HA controller
> at org.apache.hadoop.yarn.server.applicationhistoryservice.
> metrics.timeline.HBaseTimelineMetricStore.initializeSubsystem(
> HBaseTimelineMetricStore.java:118)
> at org.apache.hadoop.yarn.server.applicationhistoryservice.
> metrics.timeline.HBaseTimelineMetricStore.serviceInit(
> HBaseTimelineMetricStore.java:96)
> .
>
> The 2188 port is my setting about zookeeper, and from web UI, its status
> is OK.
> Also i check zookeeper process, it runs as usual.
>
> By the way, metrics needs listen 6188 port, but i use `netstat` to check,
> related processes do not start.
>
> I do not know how to fix it, could someone help me?
>
> Thanks
>

Re: Ranger Logging Health Check Ambari Alert

[David Quiroga]

Just to close the loop on this one. Went ahead and created a JIRA against
Ambari.

https://issues.apache.org/jira/browse/AMBARI-22708
Ranger HDFS logging health Ambari Alert


On Fri, Dec 22, 2017 at 12:16 PM, David Quiroga <quirogadf4w...@gmail.com>
wrote:

> Hello
>
> First some background:
>
> We were directed to retain audit/access records "forever" (technically 7
> years but that is basically forever in electronic log time).
>
> Each Hadoop component generates local audit logs as per their log4j
> settings. In our production system these logs would frequently fill up the
> disk. At first we would just compress them in place but that only works for
> so long and there was no redundancy with local disk storage. In others
> words, no long term plan.
>
> We started to discuss moving them to HDFS or a different storage solution.
> One of our team members pointed out the Ranger plugins are already logging
> the "same data" into HDFS.
> Probably after several meeting with the higher-ups, using Ranger logs as
> the record truth was approved. Components log4j settings were updated to
> purge data automatically.
>
> Purging local logs felt like operating with out a safety net.
> Thought it we be good to check that Ranger was successful logging to HDFS
> each day. Should mention this is a kerberized cluster, not that anything
> ever goes wrong with kerberos.
>
> Checking this would have certainly been possible with a shell script, but
> we have been pushing to centralize warning/alerts in Ambari. And so an
> Ambari alert python script to check on Ranger Logging Health was crafted.
>
> For the most part the alert was modeled after some of the hive alerts.
> At the moment it just checks that the daily /ranger/audit/ HDFS
> directory has been created.
>
> I am sure there is room for improvement but I was curious:
>
> 1. Has anyone run into this type of concern?
> a. Would an alert like this be helpful?
> b. Did you come up with another solution?
>
> 2. What is best way to get this out into the community (e.g. JIRA, if so
> Ranger or Ambari - I am checking with both mailing list)?
>   a. Any other advice on how to best share?
>
> Thank you for your time.
> -David
>

Ranger Logging Health Check Ambari Alert

[David Quiroga]

Hello

First some background:

We were directed to retain audit/access records "forever" (technically 7
years but that is basically forever in electronic log time).

Each Hadoop component generates local audit logs as per their log4j
settings. In our production system these logs would frequently fill up the
disk. At first we would just compress them in place but that only works for
so long and there was no redundancy with local disk storage. In others
words, no long term plan.

We started to discuss moving them to HDFS or a different storage solution.
One of our team members pointed out the Ranger plugins are already logging
the "same data" into HDFS.
Probably after several meeting with the higher-ups, using Ranger logs as
the record truth was approved. Components log4j settings were updated to
purge data automatically.

Purging local logs felt like operating with out a safety net.
Thought it we be good to check that Ranger was successful logging to HDFS
each day. Should mention this is a kerberized cluster, not that anything
ever goes wrong with kerberos.

Checking this would have certainly been possible with a shell script, but
we have been pushing to centralize warning/alerts in Ambari. And so an
Ambari alert python script to check on Ranger Logging Health was crafted.

For the most part the alert was modeled after some of the hive alerts.
At the moment it just checks that the daily /ranger/audit/ HDFS
directory has been created.

I am sure there is room for improvement but I was curious:

1. Has anyone run into this type of concern?
a. Would an alert like this be helpful?
b. Did you come up with another solution?

2. What is best way to get this out into the community (e.g. JIRA, if so
Ranger or Ambari - I am checking with both mailing list)?
  a. Any other advice on how to best share?

Thank you for your time.
-David