RE: Griffin - how to configure authentication for ElasticSearchentity/http request

[Lionel, Liu]

Hi Prachi and Selvaraj,

For the authentication of http requests like ES and livy, some other users also 
had such problems, we’re planning to fix them.
For the ES authentication request, it would affect in both write and read 
phase. 
There’s a PR to solve it in write phase, not merged yet, but maybe it can help 
you.
https://github.com/apache/griffin/pull/463


Thanks
Lionel, Liu

From: Prachi Kishore Hunnargikar
Sent: 2019年1月9日 16:22
To: dev@griffin.apache.org
Cc: Selvaraj K
Subject: RE: Griffin - how to configure authentication for 
ElasticSearchentity/http request

Hello Griffin community!



Has anyone tried configuring authentication for ElasticSearch entity/http 
request? We are testing Griffin on Azure HDInsight Spark cluster.



Problem


Solution


Code affected


Griffin application won’t start or even if started all jobs go in “DEAD” state


401 post to Livy error: Credentials missing or Authentication is not basic


It was failing at this line in
String resultStr = restTemplate.getForObject(uri, String.class);

JobServiceImpl.java



We added the interceptors like below and it started working:
final List<ClientHttpRequestInterceptor> interceptors = new 
ArrayList<ClientHttpRequestInterceptor>();
interceptors.add( new BasicAuthInterceptor( "xxx", "xxx" ) );
restTemplate.setInterceptors( interceptors );




service\src\main\java\org\apache\

griffin\core\job\JobServiceImpl.java

It was failing in SparkSubmitJob.java:
result = restTemplate.postForObject(livyUri,springEntity,String.class);

We added proper headers and it started working:

String auth = "xxx" + ":" + "xxx";
byte [] authentication = auth.getBytes();
byte[] base64Authentication = Base64Utils.encode(authentication);
String baseCredential = new String(base64Authentication);
headers.add(HttpHeaders.AUTHORIZATION, "Basic " + baseCredential);
headers.setAccept(Collections.singletonList(MediaType.TEXT_PLAIN));



service\src\main\java\org\apache\

griffin\core\job\SparkSubmitJob.java


Jobs run successfully but no metrics /graph is displayed upon clicking either 
“DQ Metrics, My Dashboard or Health”


ERROR 11173 --- [nio-8090-exec-3] o.a.g.c.m.MetricServiceImpl              : 
Failed to get metric values named TestJob. null

It is failing at: MetricStoreImpl.java at:
HttpEntity entity = getHttpEntityForSearch(metricName, from, size,
        tmst);

OR:

    Response response = client.performRequest("GET", urlGet,
            Collections.emptyMap(), entity);

Can anyone help in how to solve this issue?


service\src\main\java\org\apache\

griffin\core\metric\MetricStoreImpl.java




Thanks and Regards

Prachi Hunnargikar & Selvaraj K