[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 10: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 10
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 29 May 2019 08:40:50 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-29 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..

IMPALA-8560: Prometheus metrics support in Impala

-- This change adds Prometheus text explosion format metric
   generation.
-- More details can be found below:
-- https://prometheus.io/docs/instrumenting/exposition_formats
-- Added unit test to test this change

Tests:
-- Feed all this metrics to prometheus running on local host
-- Also ran it against a "./promtool" to check for any error in
   metrics format for prometheus.
Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Reviewed-on: http://gerrit.cloudera.org:8080/13345
Reviewed-by: Tim Armstrong 
Tested-by: Impala Public Jenkins 
---
M be/src/util/collection-metrics.h
M be/src/util/histogram-metric.h
M be/src/util/metrics-test.cc
M be/src/util/metrics.cc
M be/src/util/metrics.h
M tests/webserver/test_web_pages.py
6 files changed, 681 insertions(+), 0 deletions(-)

Approvals:
  Tim Armstrong: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 11
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 10:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4335/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 10
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 29 May 2019 03:14:08 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 10: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 10
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 29 May 2019 03:13:54 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 10:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3420/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 10
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 29 May 2019 02:57:26 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Harshil (Code Review)
Harshil has uploaded a new patch set (#10). ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..

IMPALA-8560: Prometheus metrics support in Impala

-- This change adds Prometheus text explosion format metric
   generation.
-- More details can be found below:
-- https://prometheus.io/docs/instrumenting/exposition_formats
-- Added unit test to test this change

Tests:
-- Feed all this metrics to prometheus running on local host
-- Also ran it against a "./promtool" to check for any error in
   metrics format for prometheus.
Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
---
M be/src/util/collection-metrics.h
M be/src/util/histogram-metric.h
M be/src/util/metrics-test.cc
M be/src/util/metrics.cc
M be/src/util/metrics.h
M tests/webserver/test_web_pages.py
6 files changed, 681 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/13345/10
--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 10
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Harshil (Code Review)
Harshil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 9:

I have added py tests


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 9
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 29 May 2019 02:15:43 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 8: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 8
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 29 May 2019 00:52:19 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 9:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4334/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 9
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 29 May 2019 00:52:37 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 9: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4334/


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 9
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 29 May 2019 00:56:23 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 9:

I should have caught this earlier, but can you add a test to test_web_pages.py 
that hits the endpoint and does some minimal sanity check (e.g. checks that a 
single metric exists). This is just to confirm that it is connected up 
end-to-end properly (we've had bugs like crashes in the past because of 
untested endpoints)


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 9
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 29 May 2019 00:56:13 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Tim Armstrong (Code Review)
Tim Armstrong has removed a vote on this change.

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Removed Code-Review+2 by Impala Public Jenkins 

--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 9
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 9: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 9
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 29 May 2019 00:52:36 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 8:

(1 comment)

I think the tests were removed because you are just not emitting property 
metrics. This seems OK, I think, but just wanted to confirm that was your 
intent before +2ing.

http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics-test.cc
File be/src/util/metrics-test.cc:

http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics-test.cc@588
PS7, Line 588: TEST_F(MetricsTest, StatsMetricsPrometheus) {
Why did you remove these tests?



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 8
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 28 May 2019 23:13:03 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Harshil (Code Review)
Harshil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics-test.cc
File be/src/util/metrics-test.cc:

http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics-test.cc@588
PS7, Line 588: TEST_F(MetricsTest, StatsMetricsPrometheus) {
> Why did you remove these tests?
Yeah, thanks for checking about this.
I removed it because code is ignoring 'property' metrics.

Since I saw some property metric were causing problems when passed to 
prometheus. Code used to convert property metric to gauge when passing to 
prometheus but gauge cannot have string values.



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 8
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 28 May 2019 23:19:56 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 8:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3415/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 8
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 28 May 2019 21:18:36 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Harshil (Code Review)
Harshil has uploaded a new patch set (#8). ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..

IMPALA-8560: Prometheus metrics support in Impala

-- This change adds Prometheus text explosion format metric
   generation.
-- More details can be found below:
-- https://prometheus.io/docs/instrumenting/exposition_formats
-- Added unit test to test this change

Tests:
-- Feed all this metrics to prometheus running on local host
-- Also ran it against a "./promtool" to check for any error in
   metrics format for prometheus.
Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
---
M be/src/util/collection-metrics.h
M be/src/util/histogram-metric.h
M be/src/util/metrics-test.cc
M be/src/util/metrics.cc
M be/src/util/metrics.h
5 files changed, 673 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/13345/8
--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 8
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Harshil (Code Review)
Harshil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 7:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics-test.cc
File be/src/util/metrics-test.cc:

http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics-test.cc@472
PS7, Line 472:   if (std::string(name) == "stats_metric" || std::string(name) 
== "histogram_metric") {
> std::string() not needed.
Done


http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics.cc
File be/src/util/metrics.cc:

http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics.cc@181
PS7, Line 181: std:
> no std:: needed
Done



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 7
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 28 May 2019 20:38:00 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 7:

(2 comments)

I think this is looking good for a first cut. Just a couple of nits to fix then 
I can +2 and start the merge.

http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics-test.cc
File be/src/util/metrics-test.cc:

http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics-test.cc@472
PS7, Line 472:   if (std::string(name) == "stats_metric" || std::string(name) 
== "histogram_metric") {
std::string() not needed.


http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics.cc
File be/src/util/metrics.cc:

http://gerrit.cloudera.org:8080/#/c/13345/7/be/src/util/metrics.cc@181
PS7, Line 181: std:
no std:: needed



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 7
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 28 May 2019 19:12:46 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 7:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3409/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 7
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 28 May 2019 17:55:21 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Harshil (Code Review)
Harshil has uploaded a new patch set (#7). ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..

IMPALA-8560: Prometheus metrics support in Impala

-- This change adds Prometheus text explosion format metric
   generation.
-- More details can be found below:
-- https://prometheus.io/docs/instrumenting/exposition_formats
-- Added unit test to test this change

Tests:
-- Feed all this metrics to prometheus running on local host
-- Also ran it against a "./promtool" to check for any error in
   metrics format for prometheus.
Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
---
M be/src/util/collection-metrics.h
M be/src/util/histogram-metric.h
M be/src/util/metrics-test.cc
M be/src/util/metrics.cc
M be/src/util/metrics.h
5 files changed, 689 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/13345/7
--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 7
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-28 Thread Harshil (Code Review)
Harshil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 6:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/13345/6/be/src/util/collection-metrics.h
File be/src/util/collection-metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/6/be/src/util/collection-metrics.h@184
PS6, Line 184:   
static_cast(boost::accumulators::min(acc_)), unit_)
> These casts all seem unnecessary - ConvertToPrometheusSecs should work for
Done


http://gerrit.cloudera.org:8080/#/c/13345/6/be/src/util/metrics.h
File be/src/util/metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/6/be/src/util/metrics.h@116
PS6, Line 116: inline
> Inline not needed.
Done


http://gerrit.cloudera.org:8080/#/c/13345/6/be/src/util/metrics.h@117
PS6, Line 117: if (type == TUnit::type::TIME_MS || type == 
TUnit::type::TIME_US
> if isn't needed - just
Done


http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h
File be/src/util/metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h@101
PS5, Line 101:   /// 
https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/
> line too long (101 > 90)
Done


http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h@112
PS5, Line 112:
> ok thanks Tim.
Done


http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h@191
PS5, Line 191: , TUnit
> TIME_MS
Done



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 6
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 28 May 2019 17:14:11 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-24 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 6:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/13345/6/be/src/util/collection-metrics.h
File be/src/util/collection-metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/6/be/src/util/collection-metrics.h@184
PS6, Line 184:   
static_cast(boost::accumulators::min(acc_)), unit_)
These casts all seem unnecessary - ConvertToPrometheusSecs should work for any 
numeric type.


http://gerrit.cloudera.org:8080/#/c/13345/6/be/src/util/metrics.h
File be/src/util/metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/6/be/src/util/metrics.h@116
PS6, Line 116: inline
Inline not needed.


http://gerrit.cloudera.org:8080/#/c/13345/6/be/src/util/metrics.h@117
PS6, Line 117: if (type == TUnit::type::TIME_MS || type == 
TUnit::type::TIME_US
if isn't needed - just

return ...;



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 6
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 25 May 2019 00:32:29 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-23 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 6:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3374/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 6
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 24 May 2019 03:01:22 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-23 Thread Harshil (Code Review)
Harshil has uploaded a new patch set (#6). ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..

IMPALA-8560: Prometheus metrics support in Impala

-- This change adds Prometheus text explosion format metric
   generation.
-- More details can be found below:
-- https://prometheus.io/docs/instrumenting/exposition_formats
-- Added unit test to test this change

Testing Done:
-- Feed all this metrics to prometheus running on local host
-- Also ran it against a "./promtool" to check for any error in
   metrics format for prometheus.
Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
---
M be/src/util/collection-metrics.h
M be/src/util/histogram-metric.h
M be/src/util/metrics-test.cc
M be/src/util/metrics.cc
M be/src/util/metrics.h
5 files changed, 696 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/13345/6
--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 6
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-22 Thread Harshil (Code Review)
Harshil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h
File be/src/util/metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h@112
PS5, Line 112:   std::string ConvertToPrometheusSecs(std::stringstream* val, 
TUnit::type unit) {
> I have two issues with the function/pattern:
ok thanks Tim.

I think I can start with your change and go ahead update other than scalar 
metric and see how it goes.



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 5
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 23 May 2019 00:21:49 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-22 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 5:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG@13
PS4, Line 13:
> I have local prometheus running on same node that is ingesting this metrics
Can you add a "Testing" section to the commit message? That'll make it more 
visible for others.


http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h
File be/src/util/metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h@112
PS5, Line 112:   std::string ConvertToPrometheusSecs(std::stringstream* val, 
TUnit::type unit) {
I have two issues with the function/pattern:

* The name of the function is confusing, since it's called for all metrics, 
including those that don't need conversion and aren't time types. I think it 
should be something that makes it more obvious at the callsite why it's called 
and what it will do (I'd suggest a name but I don't think we've decided on the 
behaviour yet).
* Converting to a string and back again is really awkward and hard to reason 
about. I'm guessing this was a workaround for C++ template hell around T, 
because T can be std::string which doesn't support a / operation.

We can solve the template thing in a cleaner way by having multiple 
implementations of a templated function, with the non-numeric ones just no-ops 
(since we don't expect the conversion to be done for non-numeric metrics).

As a proof of concept I got it working for one metric - see 
https://github.com/timarmstrong/impala/tree/prometheus


http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h@191
PS5, Line 191: TIMS_MS
TIME_MS



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 5
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 22 May 2019 20:44:11 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-21 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 5:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3313/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 5
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Wed, 22 May 2019 00:36:19 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-21 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h
File be/src/util/metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/5/be/src/util/metrics.h@101
PS5, Line 101:   /// 
https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md
line too long (101 > 90)



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 5
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 21 May 2019 23:48:42 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-21 Thread Harshil (Code Review)
Harshil has uploaded a new patch set (#5). ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..

IMPALA-8560: Prometheus metrics support in Impala

-- This change adds Prometheus text explosion format metric generation.
-- More details can be found below:
-- https://prometheus.io/docs/instrumenting/exposition_formats
-- Added unit test to test this change

Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
---
M be/src/util/collection-metrics.h
M be/src/util/histogram-metric.h
M be/src/util/metrics-test.cc
M be/src/util/metrics.cc
M be/src/util/metrics.h
5 files changed, 653 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/13345/5
--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 5
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-21 Thread Harshil (Code Review)
Harshil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 4:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG@9
PS4, Line 9: -- This change adds Prometheus text explosion format metric 
generation.
> My reading on the time units is that the prometheus convention is to conver
Done


http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG@13
PS4, Line 13:
> Did you try feeding the output of the metrics page into prometheus? I think
I have local prometheus running on same node that is ingesting this metrics and 
verified it with latest change.

Also ran it against a "./promtool" to check for any error in metrics format for 
prometheus.


http://gerrit.cloudera.org:8080/#/c/13345/4/be/src/util/metrics.h
File be/src/util/metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/4/be/src/util/metrics.h@102
PS4, Line 102:   /// name, value, human_readable, description
> Should have asked first time around, but can you include a link to https://
Done


http://gerrit.cloudera.org:8080/#/c/13345/4/be/src/util/metrics.cc
File be/src/util/metrics.cc:

http://gerrit.cloudera.org:8080/#/c/13345/4/be/src/util/metrics.cc@97
PS4, Line 97: metricsPrometheus
> metrics_prometheus to follow the same convention as other endpoints like lo
Done



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 4
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Tue, 21 May 2019 23:38:05 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-17 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG@9
PS4, Line 9: -- This change adds Prometheus text explosion format metric 
generation.
> So for 3) there is something called job_name in prometheus config, that get
My reading on the time units is that the prometheus convention is to convert 
everything to seconds. I'm not an expert though so would appreciate you 
confirming that I"m not misreading things.



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 4
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 18 May 2019 00:48:16 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-17 Thread Harshil (Code Review)
Harshil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG@9
PS4, Line 9: -- This change adds Prometheus text explosion format metric 
generation.
> I looked more at
So for 3) there is something called job_name in prometheus config, that get 
appended to metric, below has details

https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config

For 2) there is no difference in prometheus perspective between S/MS/NS, there 
is only float or int.  Can we convert all time values to either second or 
millisecond ?

Below is based on doc:
"""
value is a float represented as required by Go's ParseFloat() function. In 
addition to standard numerical values, Nan, +Inf, and -Inf are valid values 
representing not a number, positive infinity, and negative infinity, 
respectively.
The timestamp is an int64 (milliseconds since epoch, i.e. 1970-01-01 00:00:00 
UTC, excluding leap seconds), represented as required by Go's ParseInt() 
function.
"""

For 1) I will address it.



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 4
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 18 May 2019 00:39:32 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-17 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 4:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3280/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 4
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 18 May 2019 00:31:47 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-17 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3279/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 3
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 18 May 2019 00:28:34 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-17 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 4:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG@9
PS4, Line 9: -- This change adds Prometheus text explosion format metric 
generation.
I looked more at
https://prometheus.io/docs/practices/naming/ and I realised that we aren't 
following best practices. I have a few concerns to tackle either in this or a 
follow-on patch.

1. Our metric names are invalid because they include a period and hyphens: 
https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels . I 
think this needs to be tackled now since I think it probably means prometheus 
can't consume the metrics.
2. We're not indicating the units in any of the metrics. This limits the 
usefulness, and is particularly confusing for time where we have S/MS/NS. I 
think this is critical to address.
3. metrics aren't prefixed with anything that would namespace them to impala. I 
don't know if this is a major problem in prometheus (i.e. they become unusable) 
or if it's minor. We could consider prefixing everything with impala or 
apache_impala automatically.

I think we need to fix #1 before committing. It seems like converting . and - 
to _ automatically would be sufficient. #2 and #3 also need to be thought about 
and addressed before we can call the prometheus support complete.


http://gerrit.cloudera.org:8080/#/c/13345/4//COMMIT_MSG@13
PS4, Line 13:
Did you try feeding the output of the metrics page into prometheus? I think 
that would be useful validation that we got the format correct.


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc
File be/src/util/metrics-test.cc:

http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@513
PS2, Line 513:   MetricGroup metrics("CounterMetrics");
> I am not sure if I found a specific list of floating points that prometheus
It mentions that it has to be parsable by go's parsefloat function: 
https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md
 . Would be good to test edge cases like NaN, +Inf, -Inf.


http://gerrit.cloudera.org:8080/#/c/13345/4/be/src/util/metrics.h
File be/src/util/metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/4/be/src/util/metrics.h@102
PS4, Line 102:   /// name, value, human_readable, description
Should have asked first time around, but can you include a link to 
https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md
 to make it easier to find for people maintaining the code.


http://gerrit.cloudera.org:8080/#/c/13345/4/be/src/util/metrics.cc
File be/src/util/metrics.cc:

http://gerrit.cloudera.org:8080/#/c/13345/4/be/src/util/metrics.cc@97
PS4, Line 97: metricsPrometheus
metrics_prometheus to follow the same convention as other endpoints like 
log_level (we don't use camel case elsewhere)



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 4
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Sat, 18 May 2019 00:22:26 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-17 Thread Harshil (Code Review)
Harshil has uploaded a new patch set (#4). ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..

IMPALA-8560: Prometheus metrics support in Impala

-- This change adds Prometheus text explosion format metric generation.
-- More details can be found below:
 -- https://prometheus.io/docs/instrumenting/exposition_formats
-- Added unit test to test this change

Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
---
M be/src/util/collection-metrics.h
M be/src/util/histogram-metric.h
M be/src/util/metrics-test.cc
M be/src/util/metrics.cc
M be/src/util/metrics.h
5 files changed, 550 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/13345/4
--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 4
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-17 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 3:

(47 comments)

http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/collection-metrics.h
File be/src/util/collection-metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/collection-metrics.h@190
PS3, Line 190: *val << name << "_stddev " << 
std::sqrt(boost::accumulators::variance(acc_)) << "\n";
line too long (93 > 90)


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/histogram-metric.h
File be/src/util/histogram-metric.h:

http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/histogram-metric.h@75
PS3, Line 75:   std::string name, std::stringstream* value, 
std::stringstream* metric_kind) override {
line too long (92 > 90)


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc
File be/src/util/metrics-test.cc:

http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@470
PS3, Line 470:   exp_val << "# HELP " << desc << "\n" << "# TYPE " << name << " 
" << kind << "\n" << name << " " << value + "\n";
line too long (114 > 90)


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@611
PS3, Line 611:"stats_metric_last 20\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@612
PS3, Line 612:"stats_metric_min 10\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@613
PS3, Line 613:"stats_metric_max 20\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@614
PS3, Line 614:"stats_metric_mean 15\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@615
PS3, Line 615:"stats_metric_stddev 5\n",
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@630
PS3, Line 630:"stats_metric_last 2230.12\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@631
PS3, Line 631:"stats_metric_min 10\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@632
PS3, Line 632:"stats_metric_max 2230.12\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@633
PS3, Line 633:"stats_metric_mean 1120.06\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@634
PS3, Line 634:"stats_metric_stddev 1110.06\n",
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@649
PS3, Line 649:"stats_metric_last 20\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@650
PS3, Line 650:"stats_metric_min 10\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@651
PS3, Line 651:"stats_metric_max 20\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@652
PS3, Line 652:"stats_metric_mean 15\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@653
PS3, Line 653:"stats_metric_stddev 5\n",
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@668
PS3, Line 668:"stats_metric_last 20\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@669
PS3, Line 669:"stats_metric_min 10\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@670
PS3, Line 670:"stats_metric_max 20\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@671
PS3, Line 671:"stats_metric_mean 15\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@672
PS3, Line 672:"stats_metric_stddev 5\n",
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@687
PS3, Line 687:"stats_metric_last 20.567\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@688
PS3, Line 688:"stats_metric_min 10.1235\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@689
PS3, Line 689:"stats_metric_max 20.567\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@690
PS3, Line 690:"stats_metric_mean 15.3452\n"
tab used for whitespace


http://gerrit.cloudera.org:8080/#/c/13345/3/be/src/util/metrics-test.cc@691
PS3, Line 691:"stats_metric_stddev 5.22178\n",
tab used for whitespace



[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-17 Thread Harshil (Code Review)
Harshil has uploaded a new patch set (#3). ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..

IMPALA-8560: Prometheus metrics support in Impala

-- This change adds Prometheus text explosion format metric generation.
-- More details can be found below:
 -- https://prometheus.io/docs/instrumenting/exposition_formats
-- Added unit test to test this change

Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
---
M be/src/util/collection-metrics.h
M be/src/util/histogram-metric.h
M be/src/util/metrics-test.cc
M be/src/util/metrics.cc
M be/src/util/metrics.h
5 files changed, 548 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/13345/3
--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 3
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-17 Thread Harshil (Code Review)
Harshil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 2:

(12 comments)

http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/collection-metrics.h
File be/src/util/collection-metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/collection-metrics.h@80
PS2, Line 80: +
> <<
Done


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/collection-metrics.h@170
PS2, Line 170: string
> nit: std::string
Done


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/collection-metrics.h@173
PS2, Line 173: *val << name;
> You can combine these lines by chaining <<. If you do that here and below i
Done


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/histogram-metric.h
File be/src/util/histogram-metric.h:

http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/histogram-metric.h@75
PS2, Line 75: string
> nit: std::string
Done


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/histogram-metric.h@80
PS2, Line 80:   *value << "{le=0.2} ";
> Same comment about chaining << to get this into fewer lines. If you look at
Done


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc
File be/src/util/metrics-test.cc:

http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@470
PS2, Line 470: +
> nit: << instead of + here and in below lines
Done


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@488
PS2, Line 488:   AddMetricDef("gauge", TMetricKind::GAUGE, TUnit::NONE);
> Can you add tests for the supported unit types:
Done


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@495
PS2, Line 495: TEST_F(MetricsTest, PropertiesPrometheus) {
> Can you test some more edge cases with the strings, e.g. what happens if th
Done


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@513
PS2, Line 513:   AssertPrometheus(stats_val, "stats_metric",
> Do we have test coverage for all the floating point formats in the spec: ht
I am not sure if I found a specific list of floating points that prometheus 
support. Based on FAQ, all sample values are 64 bit floats.

https://prometheus.io/docs/introduction/faq/#why-are-all-sample-values-64-bit-floats-i-want-integers


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@514
PS2, Line 514:   "stats_metric_count 2\nstats_metric_last 
20.00\nstats_metric_min "
> Can you format this in a way that makes it easier to see the lines, i.e.
Done


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@533
PS2, Line 533:   "histogram-metric{le=0.2} 2500\nhistogram-metric{le=0.5} "
> Same comment on formatting.
Done


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@543
PS2, Line 543:   exp_val << "# HELP description\n# TYPE counter1 
COUNTER\ncounter1 2048\n# HELP "
> same comment on formatting.
Done



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 2
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Fri, 17 May 2019 23:30:01 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-16 Thread Tim Armstrong (Code Review)
Tim Armstrong has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 2:

(13 comments)

Some more style nits and I some requests for additional test coverage. You 
probably have a better understanding of the prometheus format than I do, so if 
you can think of more edge cases that I didn't ask for tests for, please add 
them.

http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/collection-metrics.h
File be/src/util/collection-metrics.h:

http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/collection-metrics.h@80
PS2, Line 80: +
<<

You can also just combine these lines, i.e.

  *val << name << " " << s;


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/collection-metrics.h@170
PS2, Line 170: string
nit: std::string


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/collection-metrics.h@173
PS2, Line 173: *val << name;
You can combine these lines by chaining <<. If you do that here and below it 
will be a lot more concise.


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/histogram-metric.h
File be/src/util/histogram-metric.h:

http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/histogram-metric.h@75
PS2, Line 75: string
nit: std::string


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/histogram-metric.h@80
PS2, Line 80:   *value << "{le=0.2} ";
Same comment about chaining << to get this into fewer lines. If you look at 
some of the LOG statements in the code we even have multi-line chains in a lot 
of places.


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc
File be/src/util/metrics-test.cc:

http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@470
PS2, Line 470: +
nit: << instead of + here and in below lines


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@477
PS2, Line 477: TEST_F(MetricsTest, CountersPrometheus) {
Can you add a BYTE unit counter?


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@488
PS2, Line 488:   AddMetricDef("gauge", TMetricKind::GAUGE, TUnit::NONE);
Can you add tests for the supported unit types:

  $ grep unit common/thrift/metrics.json  | sort | uniq
"units": "BYTES",
"units": "NONE",
"units": "TIME_MS",
"units": "TIME_NS",
"units": "TIME_S",
"units": "UNIT",


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@495
PS2, Line 495: TEST_F(MetricsTest, PropertiesPrometheus) {
Can you test some more edge cases with the strings, e.g. what happens if 
there's a line break in the string? It looks like the prometheus format 
requires strings to be escaped: 
https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@513
PS2, Line 513:   AssertPrometheus(stats_val, "stats_metric",
Do we have test coverage for all the floating point formats in the spec: 
https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@514
PS2, Line 514:   "stats_metric_count 2\nstats_metric_last 
20.00\nstats_metric_min "
Can you format this in a way that makes it easier to see the lines, i.e.

  "stats_metric_count 2\n"
  "stats_metric_last 20.00\n"
  "stats_metric_min 10\n"
...


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@533
PS2, Line 533:   "histogram-metric{le=0.2} 2500\nhistogram-metric{le=0.5} "
Same comment on formatting.


http://gerrit.cloudera.org:8080/#/c/13345/2/be/src/util/metrics-test.cc@543
PS2, Line 543:   exp_val << "# HELP description\n# TYPE counter1 
COUNTER\ncounter1 2048\n# HELP "
same comment on formatting.



--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 2
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 16 May 2019 20:57:28 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-16 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3255/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 2
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong 
Gerrit-Comment-Date: Thu, 16 May 2019 20:50:06 +
Gerrit-HasComments: No


[Impala-ASF-CR] IMPALA-8560: Prometheus metrics support in Impala

2019-05-16 Thread Harshil (Code Review)
Harshil has uploaded a new patch set (#2). ( 
http://gerrit.cloudera.org:8080/13345 )

Change subject: IMPALA-8560: Prometheus metrics support in Impala
..

IMPALA-8560: Prometheus metrics support in Impala

-- This change adds Prometheus text explosion format metric generation.
-- More details can be found below:
 -- https://prometheus.io/docs/instrumenting/exposition_formats
-- Added unit test to test this change

Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
---
M be/src/util/collection-metrics.h
M be/src/util/histogram-metric.h
M be/src/util/metrics-test.cc
M be/src/util/metrics.cc
M be/src/util/metrics.h
5 files changed, 280 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/13345/2
--
To view, visit http://gerrit.cloudera.org:8080/13345
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5349085a2007b568cb97f9b8130804ea64d7bb08
Gerrit-Change-Number: 13345
Gerrit-PatchSet: 2
Gerrit-Owner: Harshil 
Gerrit-Reviewer: Harshil 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Tim Armstrong