Re: JSON for mod_status

[Marion & Christophe JAILLET]

++1

:)


Author: jailletc36
Date: Fri Jan  6 07:19:20 2017
New Revision: 1777535

URL:http://svn.apache.org/viewvc?rev=1777535=rev
Log:
update

Modified:
httpd/httpd/trunk/STATUS

Modified: httpd/httpd/trunk/STATUS
URL:http://svn.apache.org/viewvc/httpd/httpd/trunk/STATUS?rev=1777535=1777534=1777535=diff
==
--- httpd/httpd/trunk/STATUS (original)
+++ httpd/httpd/trunk/STATUS Fri Jan  6 07:19:20 2017
@@ -140,6 +140,14 @@ THINGS THAT SHOULD BE CONSIDERED EARLY I
   * REST-based administration for existing (balancer/etc) and new dynamic
 runtime changes (see above)
 
+  * Improve the look of generated pages (status, load-balancer...) with dynamic

+update of the values. Generate HTML5 pages, instead of 3.2, Get rid of 
XHTML
+in the generated pages.
+
+  * Add performance monitoring of the server, of each module (?), in order to 
help
+understanding what worth looking at in order to improve overall 
performance.
+
(https://cdn.wp.nginx.com/wp-content/uploads/2016/12/Amplify-Dashboards-page-base-for-filters.png)
+
 OLD ISSUES THAT WERE THOUGHT TO BE SHOWSTOPPERS FOR 2.4 BUT OBVIOUSLY WEREN'T:
 
   * Handling of non-trailing / config by non-default handler is broken




Le 18/01/2017 à 10:56, Daniel Gruno a écrit :

On 01/17/2017 07:33 PM, Jim Jagielski wrote:

It all depends on what Bill decides regarding mod_bmx and if
it is something we intent to backport to 2.4.x

Still not sure on how to *use* BMX, or how other modules
"hook in" (right now we have several modules hook into
mod_status so the "how" is well known and documented), so
I would require some sort of docs in addition to the actual
code, of course.

Some JFDI in the meantime; https://httpd.apache.org/server-status :)
JSON: http://httpd.apache.org/server-status?view=json_status

With regards,
Daniel.


On Jan 17, 2017, at 12:35 PM, Luca Toscano  wrote:



2016-11-30 18:54 GMT+01:00 Jim Jagielski :
I'm thinking about adding JSON support to mod_status...
the "plain" version output really stinks and lacks parity
w/ the info we provide via HTML, and it would be nice
to produce a really easily parseable format.

Thoughts...?

I know it was extensively discussed, but do we have an agreement about a plan 
to add this feature?  :)

Luca

Re: JSON for mod_status

[Daniel Gruno]

On 01/18/2017 03:48 PM, Daniel Gruno wrote:
> On 01/18/2017 03:19 PM, Luca Toscano wrote:
>>
>>
>> 2017-01-18 10:56 GMT+01:00 Daniel Gruno > >:
>>
>> On 01/17/2017 07:33 PM, Jim Jagielski wrote:
>> > It all depends on what Bill decides regarding mod_bmx and if
>> > it is something we intent to backport to 2.4.x
>> >
>> > Still not sure on how to *use* BMX, or how other modules
>> > "hook in" (right now we have several modules hook into
>> > mod_status so the "how" is well known and documented), so
>> > I would require some sort of docs in addition to the actual
>> > code, of course.
>>
>> Some JFDI in the meantime; https://httpd.apache.org/server-status
>>  :)
>> JSON: http://httpd.apache.org/server-status?view=json_status
>> 
>>
>>
>> Really nice work! I'd really like to see (if the author agrees :P) some
>> mod_lua scripts shipped with the httpd release and referenced in our
>> documentation. 
> 
> Hey, it's ALv2, always has been - if we wanna ship it, so be it :)
> It's at https://github.com/Humbedooh/server-status at the moment, and
> needs a tiny bit of cleanup if it's to be shipped.

I tidied up the code and refactored the JSON format.
HTML view: http://httpd.apache.org/server-status
JSON: http://httpd.apache.org/server-status?view=json
Extended JSON:
http://httpd.apache.org/server-status?view=json=true (LOT OF
DATA :p)

Feedback/suggestions are most welcome.
I'll likely donate this to httpd after FOSDEM if there's an interest in
that.

With regards,
Daniel.

> 
> With regards,
> Daniel.
> 
>>
>> Luca
>>
> 

Re: JSON for mod_status

[William A Rowe Jr]

Really, this is now in the PMC's court. Doug and Aaron designed the BMX
bean structure and module implementation. I'm aware that jfc's crew has
also been a consumer of the module, so it already falls into that multi-vendor,
multi-use case scenario.

I'll leave this to them to advocate for httpd adoption of the work, I was just
the messenger and sometimes-maintainer.

As far as well-defined, the existing 'additonal HTML' structure for mod_status
essentially sucks; there is no way for mod_status to comprehend the data
coming back to it. It simply consists of an HTML text stream. It knows not
one thing about the extended html presented by an arbitrary third party
module.

Moving forwards with your suggestion, it is simple to pass a new 'format'
enum to the modules with the callback, so they know it should be plain
html, modern html, xml, json, or any other representation we want to add.
Modules who don't implement a given value will simply not append their
results in a particular format. So if we want to add a particularly well-
formed xml for xslt representation of status data sometime in the future,
mod_status will not know how to interpret this for other third party mods,
and those that aren't patched for the new 'format' value will simply not
add results.

The bean concept defined to mod_bmx what the data format was. I'm
not sure it was comprehensive enough in terms of arrays of arrays or
other prospective use cases. E.g. could a mod_status_bmx actually
interrogate the bmx providers and offer some legible html status
output today or xml for xslt presentation some day in the future? That's
a good question and I'd like to hear Aaron's or Doug's thoughts.

Bill




On Tue, Jan 17, 2017 at 12:33 PM, Jim Jagielski  wrote:
> It all depends on what Bill decides regarding mod_bmx and if
> it is something we intent to backport to 2.4.x
>
> Still not sure on how to *use* BMX, or how other modules
> "hook in" (right now we have several modules hook into
> mod_status so the "how" is well known and documented), so
> I would require some sort of docs in addition to the actual
> code, of course.
>
>> On Jan 17, 2017, at 12:35 PM, Luca Toscano  wrote:
>>
>>
>>
>> 2016-11-30 18:54 GMT+01:00 Jim Jagielski :
>> I'm thinking about adding JSON support to mod_status...
>> the "plain" version output really stinks and lacks parity
>> w/ the info we provide via HTML, and it would be nice
>> to produce a really easily parseable format.
>>
>> Thoughts...?
>>
>> I know it was extensively discussed, but do we have an agreement about a 
>> plan to add this feature?  :)
>>
>> Luca
>>
>>
>>
>

Re: JSON for mod_status

[Daniel Gruno]

On 01/18/2017 03:19 PM, Luca Toscano wrote:
> 
> 
> 2017-01-18 10:56 GMT+01:00 Daniel Gruno  >:
> 
> On 01/17/2017 07:33 PM, Jim Jagielski wrote:
> > It all depends on what Bill decides regarding mod_bmx and if
> > it is something we intent to backport to 2.4.x
> >
> > Still not sure on how to *use* BMX, or how other modules
> > "hook in" (right now we have several modules hook into
> > mod_status so the "how" is well known and documented), so
> > I would require some sort of docs in addition to the actual
> > code, of course.
> 
> Some JFDI in the meantime; https://httpd.apache.org/server-status
>  :)
> JSON: http://httpd.apache.org/server-status?view=json_status
> 
> 
> 
> Really nice work! I'd really like to see (if the author agrees :P) some
> mod_lua scripts shipped with the httpd release and referenced in our
> documentation. 

Hey, it's ALv2, always has been - if we wanna ship it, so be it :)
It's at https://github.com/Humbedooh/server-status at the moment, and
needs a tiny bit of cleanup if it's to be shipped.

With regards,
Daniel.

> 
> Luca
> 

Re: JSON for mod_status

[Luca Toscano]

2017-01-18 10:56 GMT+01:00 Daniel Gruno :

> On 01/17/2017 07:33 PM, Jim Jagielski wrote:
> > It all depends on what Bill decides regarding mod_bmx and if
> > it is something we intent to backport to 2.4.x
> >
> > Still not sure on how to *use* BMX, or how other modules
> > "hook in" (right now we have several modules hook into
> > mod_status so the "how" is well known and documented), so
> > I would require some sort of docs in addition to the actual
> > code, of course.
>
> Some JFDI in the meantime; https://httpd.apache.org/server-status :)
> JSON: http://httpd.apache.org/server-status?view=json_status
>
>
Really nice work! I'd really like to see (if the author agrees :P) some
mod_lua scripts shipped with the httpd release and referenced in our
documentation.

Luca

Re: JSON for mod_status

[Stefan Eissing]

Nice!

> Am 18.01.2017 um 12:46 schrieb Jim Jagielski :
> 
> This is too awesome for words :)
> 
>> On Jan 18, 2017, at 4:56 AM, Daniel Gruno  wrote:
>> 
>> On 01/17/2017 07:33 PM, Jim Jagielski wrote:
>>> It all depends on what Bill decides regarding mod_bmx and if
>>> it is something we intent to backport to 2.4.x
>>> 
>>> Still not sure on how to *use* BMX, or how other modules
>>> "hook in" (right now we have several modules hook into
>>> mod_status so the "how" is well known and documented), so
>>> I would require some sort of docs in addition to the actual
>>> code, of course.
>> 
>> Some JFDI in the meantime; https://httpd.apache.org/server-status :)
>> JSON: http://httpd.apache.org/server-status?view=json_status
>> 
>> With regards,
>> Daniel.
>> 
>>> 
 On Jan 17, 2017, at 12:35 PM, Luca Toscano  wrote:
 
 
 
 2016-11-30 18:54 GMT+01:00 Jim Jagielski :
 I'm thinking about adding JSON support to mod_status...
 the "plain" version output really stinks and lacks parity
 w/ the info we provide via HTML, and it would be nice
 to produce a really easily parseable format.
 
 Thoughts...?
 
 I know it was extensively discussed, but do we have an agreement about a 
 plan to add this feature?  :)
 
 Luca
 
 
 
>>> 
>> 
> 

Stefan Eissing

bytes GmbH
Hafenstrasse 16
48155 Münster
www.greenbytes.de

Re: JSON for mod_status

[Jim Jagielski]

This is too awesome for words :)

> On Jan 18, 2017, at 4:56 AM, Daniel Gruno  wrote:
> 
> On 01/17/2017 07:33 PM, Jim Jagielski wrote:
>> It all depends on what Bill decides regarding mod_bmx and if
>> it is something we intent to backport to 2.4.x
>> 
>> Still not sure on how to *use* BMX, or how other modules
>> "hook in" (right now we have several modules hook into
>> mod_status so the "how" is well known and documented), so
>> I would require some sort of docs in addition to the actual
>> code, of course.
> 
> Some JFDI in the meantime; https://httpd.apache.org/server-status :)
> JSON: http://httpd.apache.org/server-status?view=json_status
> 
> With regards,
> Daniel.
> 
>> 
>>> On Jan 17, 2017, at 12:35 PM, Luca Toscano  wrote:
>>> 
>>> 
>>> 
>>> 2016-11-30 18:54 GMT+01:00 Jim Jagielski :
>>> I'm thinking about adding JSON support to mod_status...
>>> the "plain" version output really stinks and lacks parity
>>> w/ the info we provide via HTML, and it would be nice
>>> to produce a really easily parseable format.
>>> 
>>> Thoughts...?
>>> 
>>> I know it was extensively discussed, but do we have an agreement about a 
>>> plan to add this feature?  :)
>>> 
>>> Luca
>>> 
>>> 
>>> 
>> 
> 

Re: JSON for mod_status

[Daniel Gruno]

On 01/17/2017 07:33 PM, Jim Jagielski wrote:
> It all depends on what Bill decides regarding mod_bmx and if
> it is something we intent to backport to 2.4.x
> 
> Still not sure on how to *use* BMX, or how other modules
> "hook in" (right now we have several modules hook into
> mod_status so the "how" is well known and documented), so
> I would require some sort of docs in addition to the actual
> code, of course.

Some JFDI in the meantime; https://httpd.apache.org/server-status :)
JSON: http://httpd.apache.org/server-status?view=json_status

With regards,
Daniel.

> 
>> On Jan 17, 2017, at 12:35 PM, Luca Toscano  wrote:
>>
>>
>>
>> 2016-11-30 18:54 GMT+01:00 Jim Jagielski :
>> I'm thinking about adding JSON support to mod_status...
>> the "plain" version output really stinks and lacks parity
>> w/ the info we provide via HTML, and it would be nice
>> to produce a really easily parseable format.
>>
>> Thoughts...?
>>
>> I know it was extensively discussed, but do we have an agreement about a 
>> plan to add this feature?  :)
>>
>> Luca
>>
>>
>>
> 

Re: JSON for mod_status

[Jim Jagielski]

It all depends on what Bill decides regarding mod_bmx and if
it is something we intent to backport to 2.4.x

Still not sure on how to *use* BMX, or how other modules
"hook in" (right now we have several modules hook into
mod_status so the "how" is well known and documented), so
I would require some sort of docs in addition to the actual
code, of course.

> On Jan 17, 2017, at 12:35 PM, Luca Toscano  wrote:
> 
> 
> 
> 2016-11-30 18:54 GMT+01:00 Jim Jagielski :
> I'm thinking about adding JSON support to mod_status...
> the "plain" version output really stinks and lacks parity
> w/ the info we provide via HTML, and it would be nice
> to produce a really easily parseable format.
> 
> Thoughts...?
> 
> I know it was extensively discussed, but do we have an agreement about a plan 
> to add this feature?  :)
> 
> Luca
> 
> 
> 

Re: JSON for mod_status

[Luca Toscano]

2016-11-30 18:54 GMT+01:00 Jim Jagielski :

> I'm thinking about adding JSON support to mod_status...
> the "plain" version output really stinks and lacks parity
> w/ the info we provide via HTML, and it would be nice
> to produce a really easily parseable format.
>
> Thoughts...?
>

I know it was extensively discussed, but do we have an agreement about a
plan to add this feature?  :)

Luca

Re: JSON for mod_status

[William A Rowe Jr]

On Fri, Dec 2, 2016 at 3:28 AM, Stefan Eissing  wrote:

>
> > Am 01.12.2016 um 23:14 schrieb Jim Jagielski :
> >
> >
> >> On Dec 1, 2016, at 2:16 PM, William A Rowe Jr 
> wrote:
> >>
> >>
> >> Note that mod_bmx_status entries focus on what most management
> >> frameworks are looking for, so thus far, it doesn't deliver an entire
> >> dataset per connection or worker thread. It certainly could, of course.
> >
> > Nor does it expose anything from any other beans created
> > by any other module.
> >
> > Can it?
> >
> > If so, exactly how.
> >
> >>
> >> Again, it simply iterates the entire contents of each bean presented
> >> to it's own print_bean_fn callback. If you have a look at;
> >> https://github.com/hyperic/mod_bmx/blob/master/modules/bmx/mod_bmx.h
> >> that should give you a somewhat higher altitude perspective on Aaron's
> >> API design.
> >
> > So mod_bmx_status has to query mod_bmx for "all beans" ??
>
> It seems to me from the name of the callback that this is for "printing"
> stuff, either all or only the name matching (which is probably a URL
> parameter. Fair enough.
>
> But as Jim pointed out, it would be more powerful if one could
> a) detect which beans are potentially available
>

+1. This is possible today simply by requesting 'all'. But an actual
query of modules and property names exposed across the entire
bean factory would be very interesting.


> b) retrieve and aggregate them
>

We can on a 1:1 basis today (either an entire module scope, or only
named properties across all modules, or all of the above.)

It would be nice to enhance this as a many:1 response of several
targeted data points desired.

So, we'd have BeanProviders that take over a namespace. They register at
> server start.
>
> ProviderRegistry["proxy/balancer"] -> registered provider hook by the
> module
> ProviderRegistry["proxy/type/h2"]-> registered provider hook by
> mod_proxy_http2
> ProviderRegistry["proxy/type/h2c"]   -> registered provider hook by
> mod_proxy_http2
>
> One can query this registry.
>

One would also register the Property names exposed by the provider, no?


> For lookups, there is a BeanContext that caches beans once retrieved. So
> there is not need to look them up again in the same request. That is passed
> to all registered bean provider queries, so that providers can also query
> for other beans while assembling their own.
>
> A bean produced this way is then send to the client in the requested
> format using a standard set of BeanToJSON, BeanToXML, BeanToHTML
> serializers. The beans just need to be a slim wrapper around
> apr_table_t/apr_array_header_t/const char*/int/apr_size_t/apr_off_
> t/double.
>
> The registry can be mapped to URL space, so
>  * https://localhost/status/proxy/balancer would list the bean there,
>  * https://localhost/status/proxy/type would list all beans registered
> with that prefix.
>

The current syntax is;

https://localhost/status/?proxy_balancer
;*
https://localhost/status/?proxy_balancer
;Pool=app1,State=Error

etc.

I'd have no issue with enhancing this to support your proposed syntax.
As the already adopted backend of one or more management agents,
I'd like to see us continue to accept the existing syntax. Use whichever
is most appropriate.

The existing design expects the provider to choose a name once and
remain consistent. So even as new property/value pairs can be added,
the behavior of existing ones remains consistent. Otherwise the interface
would have no value to a management agent / snmp replacement.

Bill

Re: JSON for mod_status

[William A Rowe Jr]

On Thu, Dec 1, 2016 at 4:14 PM, Jim Jagielski  wrote:

>
> > On Dec 1, 2016, at 2:16 PM, William A Rowe Jr 
> wrote:
> >
> >
> > Note that mod_bmx_status entries focus on what most management
> > frameworks are looking for, so thus far, it doesn't deliver an entire
> > dataset per connection or worker thread. It certainly could, of course.
>
> Nor does it expose anything from any other beans created
> by any other module.
>
> Can it?
>
> If so, exactly how.
>
> >
> > Again, it simply iterates the entire contents of each bean presented
> > to it's own print_bean_fn callback. If you have a look at;
> > https://github.com/hyperic/mod_bmx/blob/master/modules/bmx/mod_bmx.h
> > that should give you a somewhat higher altitude perspective on Aaron's
> > API design.
>
> So mod_bmx_status has to query mod_bmx for "all beans" ??
>

You are still thinking upside down. Think of mod_bmx as the data sink
and mod_bmx_status as the data source. mod_bmx_status doesn't look
at anything except the internals of httpd. Nobody queries mod_bmx_status
directly except mod_bmx. The user agent queries mod_bmx.

Re: JSON for mod_status

[Stefan Eissing]

> Am 01.12.2016 um 23:14 schrieb Jim Jagielski :
> 
> 
>> On Dec 1, 2016, at 2:16 PM, William A Rowe Jr  wrote:
>> 
>> 
>> Note that mod_bmx_status entries focus on what most management
>> frameworks are looking for, so thus far, it doesn't deliver an entire
>> dataset per connection or worker thread. It certainly could, of course.
> 
> Nor does it expose anything from any other beans created
> by any other module.
> 
> Can it?
> 
> If so, exactly how.
> 
>> 
>> Again, it simply iterates the entire contents of each bean presented
>> to it's own print_bean_fn callback. If you have a look at;
>> https://github.com/hyperic/mod_bmx/blob/master/modules/bmx/mod_bmx.h
>> that should give you a somewhat higher altitude perspective on Aaron's
>> API design.
> 
> So mod_bmx_status has to query mod_bmx for "all beans" ??

It seems to me from the name of the callback that this is for "printing" stuff, 
either all or only the name matching (which is probably a URL parameter. Fair 
enough. 

But as Jim pointed out, it would be more powerful if one could
a) detect which beans are potentially available
b) retrieve and aggregate them

So, we'd have BeanProviders that take over a namespace. They register at server 
start.

ProviderRegistry["proxy/balancer"] -> registered provider hook by the module
ProviderRegistry["proxy/type/h2"]-> registered provider hook by 
mod_proxy_http2
ProviderRegistry["proxy/type/h2c"]   -> registered provider hook by 
mod_proxy_http2

One can query this registry.

For lookups, there is a BeanContext that caches beans once retrieved. So there 
is not need to look them up again in the same request. That is passed to all 
registered bean provider queries, so that providers can also query for other 
beans while assembling their own.

A bean produced this way is then send to the client in the requested format 
using a standard set of BeanToJSON, BeanToXML, BeanToHTML serializers. The 
beans just need to be a slim wrapper around 
apr_table_t/apr_array_header_t/const char*/int/apr_size_t/apr_off_t/double.

The registry can be mapped to URL space, so 
 * https://localhost/status/proxy/balancer would list the bean there,
 * https://localhost/status/proxy/type would list all beans registered with 
that prefix.

-Stefan

Re: JSON for mod_status

[Stefan Eissing]

> Am 02.12.2016 um 06:06 schrieb PKU.孙斌 :
> 
> As to accept headers, that's a pain in the butt if someone just wants to grab 
> a copy of the data for debugging a custom parser or something.  So much 
> easier to just wget BLAH than to mess with the accept headers on any utility 
> we want to fetch with.

I see the point of Accept-* header triggering representation, but there should 
always be a URL for each format that sends that one format all the time.

-Stefan

RE: JSON for mod_status

[PKU . 孙斌]

Since JSON is now so important to the Web client-end world, it seems that Httpd 
would be better to support JSON data i/o filtering with a JSON parser 
integrated, or put it into the Libapr?

Bing

---
From: dev-return-87644-bswen=pku.edu...@httpd.apache.org 
[mailto:dev-return-87644-bswen=pku.edu...@httpd.apache.org] 代表 Houser, Rick

Personally, I'd rather have XML, but it may make sense to support multiple 
machine readable formats down the line.  As such, using a new parameter for 
every possibility gets messy.  If this gets implemented as a get parameter, how 
about making one parameter with multiple potential values down the line?  It's 
not like they would all have to be implemented up-front.

?type=json
?type=xml
?type=js
?type=csv
Etc.

As to accept headers, that's a pain in the butt if someone just wants to grab a 
copy of the data for debugging a custom parser or something.  So much easier to 
just wget BLAH than to mess with the accept headers on any utility we want to 
fetch with.


Rick Houser
Web Administration

> -Original Message-
> From: Jordan Gigov [mailto:colad...@gmail.com]
> Sent: Wednesday, November 30, 2016 13:20
> To: dev@httpd.apache.org
> Subject: Re: JSON for mod_status
> 
> I think a better approach than "?json=true" would be to to respect the
> "Accept" header values of "application/json" and "text/json" if they
> have the higher priority.
> XML output should also be an option, if it will indeed serve as an API
> end-point.
> 
> On 30 November 2016 at 20:08, Luca Toscano <toscano.l...@gmail.com>
> wrote:
> >
> >
> > 2016-11-30 19:03 GMT+01:00 Daniel Gruno <humbed...@apache.org>:
> >>
> >> On 11/30/2016 06:54 PM, Jim Jagielski wrote:
> >> > I'm thinking about adding JSON support to mod_status...
> >> > the "plain" version output really stinks and lacks parity
> >> > w/ the info we provide via HTML, and it would be nice
> >> > to produce a really easily parseable format.
> >> >
> >> > Thoughts...?
> >
> >
> > +1, this will simplify a lot things like metrics collectors polling httpd
> > regularly (that don't need a nice html format).
> >
> >> +1, a ?json=true or some such to the query would be helpful.
> >> so you could do
> >> /server-status?json=true
> >> or
> >> /server-status?auto=true
> >>
> >
> > +1, I like the idea!
> >
> > Thanks!
> >
> > Luca
> >

Re: JSON for mod_status

[Jim Jagielski]

> On Dec 1, 2016, at 2:16 PM, William A Rowe Jr  wrote:
> 
> 
> Note that mod_bmx_status entries focus on what most management
> frameworks are looking for, so thus far, it doesn't deliver an entire
> dataset per connection or worker thread. It certainly could, of course.

Nor does it expose anything from any other beans created
by any other module.

Can it?

If so, exactly how.

> 
> Again, it simply iterates the entire contents of each bean presented
> to it's own print_bean_fn callback. If you have a look at;
> https://github.com/hyperic/mod_bmx/blob/master/modules/bmx/mod_bmx.h
> that should give you a somewhat higher altitude perspective on Aaron's
> API design.

So mod_bmx_status has to query mod_bmx for "all beans" ??

Re: JSON for mod_status

[William A Rowe Jr]

On Thu, Dec 1, 2016 at 12:33 PM, Jim Jagielski  wrote:

>
> > On Dec 1, 2016, at 12:53 PM, William A Rowe Jr 
> wrote:
> >
> > Finally the query fn in mod_bmx_status performs the callback indicated
> > through its invocation to unspool the data in presentation format, which
> > lives back in mod_bmx and behaves identically for every bmx extension.
> > Adding the properties by type takes all of the onus off of each extension
> > to know how to represent them once stored. E.g. our query_hook function
> > can be as simple as;
> > /* create the bean */
> > bmx_bean_create(_status_bean, bmx_status_objectname, r->pool);
> >
> > bmx_bean_prop_add(bmx_status_bean,
> > bmx_property_string_create("ServerName",
> >ap_get_server_name(r),
> >r->pool));print_bean_fn(r,
> bmx_status_bean);
> >
> >
>
> OK, this is the point I have the question about. From what
> I can see, mod_bmx_status creates bmx_status_bean, adds stuff
> to it, and then prints it out.
>
> But how does it get access to, for example, the beans created and
> populated in mod_bmx_vhost? I understand that if you KNOW what
> beans you are looking for, you can query them via mod_bmx, but
> how do you know what beans have been added.
>

The beans are individually reflected to the print_bean_fn by each of
the consumers who registered against the query_hook.

mod_bmx then iterates the name of the bean and the property names
to filter out what is wanted. If *everything* is wanted, simply don't give
any query args to the bmx-handler request.


> Right now, for example, we have a simple hook that mod_socache_redis
> (for example) hooks into and once mod_status is done, it loops
> through all registered hooks. But I don't see any similar type of
> functionality in mod_bmx_status which allows mod_bmx_status to
> reproduce the full status information that mod_status currently
> does.
>

Note that mod_bmx_status entries focus on what most management
frameworks are looking for, so thus far, it doesn't deliver an entire
dataset per connection or worker thread. It certainly could, of course.

One thing not implemented, json (or even bean) arrays, which might
become very useful for tabular data from status, balancer, etc.

So yeah, other modules create their beans. Fine. But how
> do those beans get "registered" in mod_bmx_status so that they
> can actually be displayed?


Again, it simply iterates the entire contents of each bean presented
to it's own print_bean_fn callback. If you have a look at;
https://github.com/hyperic/mod_bmx/blob/master/modules/bmx/mod_bmx.h
that should give you a somewhat higher altitude perspective on Aaron's
API design.

More background;
https://github.com/hyperic/mod_bmx/blob/master/README-BMX.txt

Re: JSON for mod_status

[Jim Jagielski]

> On Dec 1, 2016, at 12:53 PM, William A Rowe Jr  wrote:
> 
> 
> Finally the query fn in mod_bmx_status performs the callback indicated
> through its invocation to unspool the data in presentation format, which
> lives back in mod_bmx and behaves identically for every bmx extension.
> Adding the properties by type takes all of the onus off of each extension
> to know how to represent them once stored. E.g. our query_hook function
> can be as simple as;
> /* create the bean */
> bmx_bean_create(_status_bean, bmx_status_objectname, r->pool);
> 
> bmx_bean_prop_add(bmx_status_bean,
> bmx_property_string_create("ServerName",
>ap_get_server_name(r),
>r->pool));print_bean_fn(r, 
> bmx_status_bean);
> 
> 

OK, this is the point I have the question about. From what
I can see, mod_bmx_status creates bmx_status_bean, adds stuff
to it, and then prints it out.

But how does it get access to, for example, the beans created and
populated in mod_bmx_vhost? I understand that if you KNOW what
beans you are looking for, you can query them via mod_bmx, but
how do you know what beans have been added.

Right now, for example, we have a simple hook that mod_socache_redis
(for example) hooks into and once mod_status is done, it loops
through all registered hooks. But I don't see any similar type of
functionality in mod_bmx_status which allows mod_bmx_status to
reproduce the full status information that mod_status currently
does.

So yeah, other modules create their beans. Fine. But how
do those beans get "registered" in mod_bmx_status so that they
can actually be displayed?

Re: JSON for mod_status

[William A Rowe Jr]

On Thu, Dec 1, 2016 at 8:08 AM, Jim Jagielski  wrote:

> My question is how do the beans, for example, from mod_bmx_vhost
> get displayed via mod_bmx_status?
>
> I understand that one queries mod_bmx for specific beans, but at
> the end of the day people still want/need a comprehensize
> mod_status-like display. My expectation would be that mod_bmx_status
> would provide that.
>


mod_bmx is the hub module which handles the bmx-handler generator.
Configuration is as straightforward as compiling in or loading mod_bmx.so
and following a typical server-status handler config;


SetHandler bmx-handler
Require ip 127.0.0.1


What mod_bmx reports depends on who hooks into it. You can run it
by only loading mod_bmx_status, or only mod_bmx_vhost, or any
combination of multiple providers. An obvious new extension would
be mod_proxy_balancer pool load reporting.

mod_bmx_status hooks into this framework by setting a server-lifetime
bean indicating that ServerStatus is extended or not, and then hooks
into the mod_bmx query hook during the pre-config phase;

APR_OPTIONAL_HOOK(bmx, query_hook, bmx_status_query_hook, NULL, NULL,
  APR_HOOK_MIDDLE);

Our mod_bmx_status query_hook handler is a fairly typical 'handler',
of data instead of content;

static int bmx_status_query_hook(request_rec *r,
 const struct bmx_objectname *query,
 bmx_bean_print print_bean_fn)


We don't want to refresh our data if the query indicates that the request
to the bmx-handler excluded server-status data, so there's a short circuit
right off the bat. Provided some subset of server-status data is needed,
this function continues by populating the data for mod_bmx to present
(which it can further filter by item name without help from the extensions).

Finally the query fn in mod_bmx_status performs the callback indicated
through its invocation to unspool the data in presentation format, which
lives back in mod_bmx and behaves identically for every bmx extension.
Adding the properties by type takes all of the onus off of each extension
to know how to represent them once stored. E.g. our query_hook function
can be as simple as;

/* create the bean */
bmx_bean_create(_status_bean, bmx_status_objectname, r->pool);

bmx_bean_prop_add(bmx_status_bean,
bmx_property_string_create("ServerName",
   ap_get_server_name(r),
   r->pool));print_bean_fn(r,
bmx_status_bean);


For Stefan's and others concerns, it would probably be most useful
for Doug or Aaron to explain how they came to the bean datum model
for representing bmx's results.



.

Re: JSON for mod_status

[Jim Jagielski]

My question is how do the beans, for example, from mod_bmx_vhost
get displayed via mod_bmx_status?

I understand that one queries mod_bmx for specific beans, but at
the end of the day people still want/need a comprehensize
mod_status-like display. My expectation would be that mod_bmx_status
would provide that.

> On Nov 30, 2016, at 5:01 PM, William A Rowe Jr  wrote:
> 
> You are looking at one fold of monitoring data. See, for example, 
> mod_bmx_vhost which further extends many beans by hostname.
> 
> The response generator and framework lives in mod_bmx itself.
> 
> 
> On Nov 30, 2016 14:35, "Jim Jagielski"  wrote:
> One thing that I can't understand from an initial look
> is how the whole hook thing is. In mod_status, we have a hook
> that is run in mod_status and other modules use to supplement
> the mod_status output.
> 
> With mod_bmx it looks like instead whatever "chunk" of data
> you want to make available, you put into a bean, but mod_bmx_status
> uses a local bean with no provision to print out other
> beans as well.
> 
> How does one "extend" the info printed by mod_bmx_status
> via other beans added and populated by other modules?
> 
> 

Re: JSON for mod_status

[Stefan Eissing]

First: +1 to have machine-readable output from mod_status, maybe we'll even add 
test cases!

Second: -1 to have various hooks/sub-modules serialize JSON by themselves (or 
other formats). Instead, I'd rather have us chose a data/bean/binary json API 
with which to construct objects to pass around and have a standard set of 
serializers/parsers for JSON, XML, HTML, etc.

As to passing such data around, I've made very good experiences with wrapping 
response status/headers into their own meta bucket type for HTTP/2. That way 
the raw data gets passed around as buckets/through brigades and the main 
connection can serialize that to HTTP/2 frames.

-Stefan

> Am 30.11.2016 um 21:34 schrieb Jim Jagielski :
> 
> One thing that I can't understand from an initial look
> is how the whole hook thing is. In mod_status, we have a hook
> that is run in mod_status and other modules use to supplement
> the mod_status output.
> 
> With mod_bmx it looks like instead whatever "chunk" of data
> you want to make available, you put into a bean, but mod_bmx_status
> uses a local bean with no provision to print out other
> beans as well.
> 
> How does one "extend" the info printed by mod_bmx_status
> via other beans added and populated by other modules?
> 

Re: JSON for mod_status

[William A Rowe Jr]

You are looking at one fold of monitoring data. See, for example,
mod_bmx_vhost which further extends many beans by hostname.

The response generator and framework lives in mod_bmx itself.

On Nov 30, 2016 14:35, "Jim Jagielski"  wrote:

One thing that I can't understand from an initial look
is how the whole hook thing is. In mod_status, we have a hook
that is run in mod_status and other modules use to supplement
the mod_status output.

With mod_bmx it looks like instead whatever "chunk" of data
you want to make available, you put into a bean, but mod_bmx_status
uses a local bean with no provision to print out other
beans as well.

How does one "extend" the info printed by mod_bmx_status
via other beans added and populated by other modules?

RE: JSON for mod_status

[Houser, Rick]

Personally, I'd rather have XML, but it may make sense to support multiple 
machine readable formats down the line.  As such, using a new parameter for 
every possibility gets messy.  If this gets implemented as a get parameter, how 
about making one parameter with multiple potential values down the line?  It's 
not like they would all have to be implemented up-front.

?type=json
?type=xml
?type=js
?type=csv
Etc.

As to accept headers, that's a pain in the butt if someone just wants to grab a 
copy of the data for debugging a custom parser or something.  So much easier to 
just wget BLAH than to mess with the accept headers on any utility we want to 
fetch with.


Rick Houser
Web Administration

> -Original Message-
> From: Jordan Gigov [mailto:colad...@gmail.com]
> Sent: Wednesday, November 30, 2016 13:20
> To: dev@httpd.apache.org
> Subject: Re: JSON for mod_status
> 
> I think a better approach than "?json=true" would be to to respect the
> "Accept" header values of "application/json" and "text/json" if they
> have the higher priority.
> XML output should also be an option, if it will indeed serve as an API
> end-point.
> 
> On 30 November 2016 at 20:08, Luca Toscano <toscano.l...@gmail.com>
> wrote:
> >
> >
> > 2016-11-30 19:03 GMT+01:00 Daniel Gruno <humbed...@apache.org>:
> >>
> >> On 11/30/2016 06:54 PM, Jim Jagielski wrote:
> >> > I'm thinking about adding JSON support to mod_status...
> >> > the "plain" version output really stinks and lacks parity
> >> > w/ the info we provide via HTML, and it would be nice
> >> > to produce a really easily parseable format.
> >> >
> >> > Thoughts...?
> >
> >
> > +1, this will simplify a lot things like metrics collectors polling httpd
> > regularly (that don't need a nice html format).
> >
> >> +1, a ?json=true or some such to the query would be helpful.
> >> so you could do
> >> /server-status?json=true
> >> or
> >> /server-status?auto=true
> >>
> >
> > +1, I like the idea!
> >
> > Thanks!
> >
> > Luca
> >

Re: JSON for mod_status

[Jim Jagielski]

One thing that I can't understand from an initial look
is how the whole hook thing is. In mod_status, we have a hook
that is run in mod_status and other modules use to supplement
the mod_status output.

With mod_bmx it looks like instead whatever "chunk" of data
you want to make available, you put into a bean, but mod_bmx_status
uses a local bean with no provision to print out other
beans as well.

How does one "extend" the info printed by mod_bmx_status
via other beans added and populated by other modules?

Re: JSON for mod_status

[William A Rowe Jr]

On Wed, Nov 30, 2016 at 2:10 PM, Jim Jagielski  wrote:

> One final note:
>
> The reason I brought all this up is that right now, using mod_status
> for actual automated status checking is downright painful. At
> ApacheCon, due to, I guess, me adding the mod_status hooks to
> the redis and memcache socache providers, people came up to
> me and asked when, if ever, we would support JSON in mod_status.
>

++1 :)

S
>
> What I was hoping for was an implementation that could easily,
> and in some reasonable timeframe, be made available to our
> 2.4.x community. I don't want something that just ends up
> in trunk, to sit and languish until the day when we eventually
> decide on releasing 2.6/3.0.
>

Since it's written and is working just fine (and is part of the RHEL
distribution) - let's bring bmx in, then think about how to refactor
the bean data and status hooks to converge them in 2.next/3.0.
Or we can play NIH. (Especially absurd since an httpd committer
wrote the whole first implementation.)

Re: JSON for mod_status

[Jim Jagielski]

One final note:

The reason I brought all this up is that right now, using mod_status
for actual automated status checking is downright painful. At
ApacheCon, due to, I guess, me adding the mod_status hooks to
the redis and memcache socache providers, people came up to
me and asked when, if ever, we would support JSON in mod_status.

S

What I was hoping for was an implementation that could easily,
and in some reasonable timeframe, be made available to our
2.4.x community. I don't want something that just ends up
in trunk, to sit and languish until the day when we eventually
decide on releasing 2.6/3.0.

Re: JSON for mod_status

[William A Rowe Jr]

On Wed, Nov 30, 2016 at 1:58 PM, Jim Jagielski  wrote:

> I'm not pushing back on mod_bmx at all... I just think
> that there is a better understanding here about "mod_status
> producing JSON" than what the changeover to mod_bmx would
> be.
>
> What I would push back on would be having 2 implementations,
> since that's just weird :) But if the BMX data format is
> "better" then we should use that.
>
> As Nick said, what is key is that we should produce this
> internal data in a known and easily parseable exchange
> format, one that we can then directly send as a response
> as well as one which we can, via a provider-like interface,
> xlate to HTML and other "common" formats. Right now, I
> think "plain", "json" and "html" are the only ones we
> should worry about.
>

So to be clear, it is easy to flip mod_status's core data to any
one of dozens of formats without any further debate.

It is not so easy to carry along the third party hooked status
representations of *their* data, not being able to predict how
httpd 2.4.x will keep changing behavior during it's 'maintenance'
lifespan, not knowing how they presented  lists or tables
and how they aught to be mapped to json tokens.

Contra-wise, turning everything into json-always with 2.next
or 3.0 isn't a solution either, since there's no way for mod_status
to predict the best representation of the json output from any
hooked third party modules.

So the best solution, IMO, is to map as beans all of the data
to be represented as raw json (and other raw formats), which can
be filtered for the 'interesting' data that downstream monitoring
clients want to consume, and leave the existing status hook
for third party modules to map their data into presentation html
(or xml, if we want to go that way.)

Re: JSON for mod_status

[Jim Jagielski]

I'm not pushing back on mod_bmx at all... I just think
that there is a better understanding here about "mod_status
producing JSON" than what the changeover to mod_bmx would
be.

What I would push back on would be having 2 implementations,
since that's just weird :) But if the BMX data format is
"better" then we should use that.

As Nick said, what is key is that we should produce this
internal data in a known and easily parseable exchange
format, one that we can then directly send as a response
as well as one which we can, via a provider-like interface,
xlate to HTML and other "common" formats. Right now, I
think "plain", "json" and "html" are the only ones we
should worry about.

Re: JSON for mod_status

[William A Rowe Jr]

On Wed, Nov 30, 2016 at 1:41 PM, Jim Jagielski  wrote:

> > On Nov 30, 2016, at 2:31 PM, William A Rowe Jr 
> wrote:
> >
> > On Wed, Nov 30, 2016 at 1:27 PM, Jim Jagielski  wrote:
> >
> > > On Nov 30, 2016, at 1:56 PM, William A Rowe Jr 
> wrote:
> > >
> > >
> > > There's no way to anticipate the "right way" to map json
> > > tables to presentation-level results.
> >
> > ???
> >
> > Certainly that is the job of the xlate provider that
> > would be used, wouldn't it. Or are you suggesting that
> > JSON itself isn't suitable as a data format in this case?
> >
> > I'm asking how one would maintain the xslt for the half dozen
> > json data providers of mod_status extensions within the stock
> > distro, PLUS any mod_status extension providers in the third
> > party world of modules?
> >
>
> U the data providers that hook into mod_status simply
> push JSON. Same w/ any other extension providers. That is,
> ALL mod_status does is provide JSON data; mod_status_html takes
> that JSON format and xlates it to HTML; mod_status_plain takes
> it and xlates it to simple text.
>
> Where does JSON fail and bean succeed?
>
> Other than being something that I brought up, I'm not sure
> I understand your resistance, or why/how BMX is "better"
> than JSON.


BMX is entireliy JSON. I have no argument with your suggestion, which
is why I'm bringing it up (you are the only one on the private list to have
pushed back against adopting mod_bmx, why so I'm really not clear.)

As you can see in the sources, mod_bmx_status is interrogating the
scoreboard, not mod_status, for the underlying data.

Beans provide a completely extensible way to map additional data for
mod_bmx to present. What I'm questioning is whether a transliteration
of unknown-future data can ever succeed, or whether third party modules
won't need both a presentation-layer and json/bean-layer to represent
their data in user and machine usable formats.

I'm not familiar with a good interface for json -> html, pointers would be
appreciated. I'm fairly familiar with xml -> json + xml -> html using xslt.
That's presuming you know the entire plane of data to represent.

Re: JSON for mod_status

[Jim Jagielski]

So the only concern is that you prefer XML over JSON??

Re: JSON for mod_status

[Jim Jagielski]

> On Nov 30, 2016, at 2:31 PM, William A Rowe Jr  wrote:
> 
> On Wed, Nov 30, 2016 at 1:27 PM, Jim Jagielski  wrote:
> 
> > On Nov 30, 2016, at 1:56 PM, William A Rowe Jr  wrote:
> >
> >
> > There's no way to anticipate the "right way" to map json
> > tables to presentation-level results.
> 
> ???
> 
> Certainly that is the job of the xlate provider that
> would be used, wouldn't it. Or are you suggesting that
> JSON itself isn't suitable as a data format in this case?
> 
> I'm asking how one would maintain the xslt for the half dozen
> json data providers of mod_status extensions within the stock 
> distro, PLUS any mod_status extension providers in the third
> party world of modules?
> 

U the data providers that hook into mod_status simply
push JSON. Same w/ any other extension providers. That is,
ALL mod_status does is provide JSON data; mod_status_html takes
that JSON format and xlates it to HTML; mod_status_plain takes
it and xlates it to simple text.

Where does JSON fail and bean succeed?

Other than being something that I brought up, I'm not sure
I understand your resistance, or why/how BMX is "better"
than JSON.

Re: JSON for mod_status

[William A Rowe Jr]

On Wed, Nov 30, 2016 at 1:27 PM, Jim Jagielski  wrote:

>
> > On Nov 30, 2016, at 1:56 PM, William A Rowe Jr 
> wrote:
> >
> >
> > There's no way to anticipate the "right way" to map json
> > tables to presentation-level results.
>
> ???
>
> Certainly that is the job of the xlate provider that
> would be used, wouldn't it. Or are you suggesting that
> JSON itself isn't suitable as a data format in this case?
>

If we have well-formed xml today out of mod_status, it's pretty
straightforward to write the json transformation you want as an
xslt through filtering, provided you plug in a libxslt transform.

Re: JSON for mod_status

[William A Rowe Jr]

On Wed, Nov 30, 2016 at 1:27 PM, Jim Jagielski  wrote:

>
> > On Nov 30, 2016, at 1:56 PM, William A Rowe Jr 
> wrote:
> >
> >
> > There's no way to anticipate the "right way" to map json
> > tables to presentation-level results.
>
> ???
>
> Certainly that is the job of the xlate provider that
> would be used, wouldn't it. Or are you suggesting that
> JSON itself isn't suitable as a data format in this case?
>

I'm asking how one would maintain the xslt for the half dozen
json data providers of mod_status extensions within the stock
distro, PLUS any mod_status extension providers in the third
party world of modules?

Re: JSON for mod_status

[William A Rowe Jr]

On Wed, Nov 30, 2016 at 1:22 PM, Jim Jagielski  wrote:

> Any online examples of it running? It's been a SUPER long
> time since I looked at it. And I can't recall what the bean
> framework is or does...


As with /server-status - not advised unless you EDONTCARE about
host inspection.

The bean framework allows you to expand the nodes known to the
bmx data store, with new arbitrary fields and values.

This 380-line module maps all of the conventional mod_status data
into the status bean data for reporting through mod_bmx...

https://github.com/hyperic/mod_bmx/blob/master/modules/bmx/mod_bmx_status.c

Other modules can provide similar beans that can be globbed from
the /bmx-status generated page based on pattern matching. There
are few predefined constraints on the labeling.

Re: JSON for mod_status

[Jim Jagielski]

> On Nov 30, 2016, at 1:56 PM, William A Rowe Jr  wrote:
> 
> 
> There's no way to anticipate the "right way" to map json
> tables to presentation-level results.

???

Certainly that is the job of the xlate provider that
would be used, wouldn't it. Or are you suggesting that
JSON itself isn't suitable as a data format in this case?

Re: JSON for mod_status

[Kurt Newman]

Sounds like a great module.

Couple questions.
1. This seems to provide a lot more information than mod_status.  Is this 
correct?
2. If this provides more information than mod_status, what are the implications 
of running this on a shared server where local users can query Apache directly 
via localhost, and you may not want them to?
3. What happens with the permissions of the backend DBM file when each process 
is running as a different user (e.g. via mod_ruid2 or mpm_itk)?

> On Nov 30, 2016, at 12:50 PM, William A Rowe Jr  wrote:
> 
> On Wed, Nov 30, 2016 at 11:54 AM, Jim Jagielski  > wrote:
> I'm thinking about adding JSON support to mod_status...
> the "plain" version output really stinks and lacks parity
> w/ the info we provide via HTML, and it would be nice
> to produce a really easily parseable format.
> 
> Thoughts...?
> 
> It's right here...
> 
> https://github.com/hyperic/mod_bmx/blob/master/README-BMX.txt 
> 
> 
> I'd brought this to the PMC's attention to see if there were sufficient
> maintainers and interested parties. In short, the original author was
> our own Bannert.
> 
> In short, Pivotal is offering this to the HTTPD project under AL 2.0
> (as it's already licensed).
> 
> Right now, jfclere and I are the only two really looking at the code,
> although it has many commercial adopters (the preferred httpd
> introspection agent of the Hyperic monitoring software).
> 
> What bannert coded was more than converting status -> json.
> He build a bean framework that is extensible to any other httpd
> module. It seems like that bean framework might even be a
> candidate to move into the httpd core with 2.next?
> 
> As far as extentions, his most useful was the mod_bmx_vhost
> plug in. Taking it to the next step, that module should be able
> to shoot out a nice presentation table for mod_status users,
> while folks inspecting the bmx json bean interface already get
> these vhost-specific breakdowns.
> 
> 
> 
> 



smime.p7s
Description: S/MIME cryptographic signature

Re: JSON for mod_status

[Jim Jagielski]

Any online examples of it running? It's been a SUPER long
time since I looked at it. And I can't recall what the bean
framework is or does...

> On Nov 30, 2016, at 1:50 PM, William A Rowe Jr  wrote:
> 
> On Wed, Nov 30, 2016 at 11:54 AM, Jim Jagielski  wrote:
> I'm thinking about adding JSON support to mod_status...
> the "plain" version output really stinks and lacks parity
> w/ the info we provide via HTML, and it would be nice
> to produce a really easily parseable format.
> 
> Thoughts...?
> 
> It's right here...
> 
> https://github.com/hyperic/mod_bmx/blob/master/README-BMX.txt
> 
> I'd brought this to the PMC's attention to see if there were sufficient
> maintainers and interested parties. In short, the original author was
> our own Bannert.
> 
> In short, Pivotal is offering this to the HTTPD project under AL 2.0
> (as it's already licensed).
> 
> Right now, jfclere and I are the only two really looking at the code,
> although it has many commercial adopters (the preferred httpd
> introspection agent of the Hyperic monitoring software).
> 
> What bannert coded was more than converting status -> json.
> He build a bean framework that is extensible to any other httpd
> module. It seems like that bean framework might even be a
> candidate to move into the httpd core with 2.next?
> 
> As far as extentions, his most useful was the mod_bmx_vhost
> plug in. Taking it to the next step, that module should be able
> to shoot out a nice presentation table for mod_status users,
> while folks inspecting the bmx json bean interface already get
> these vhost-specific breakdowns.
> 
> 
> 
> 

Re: JSON for mod_status

[Jim Jagielski]

Yeah, this is the real solution... mod_status should produce
just JSON and there are addable filters (or providers) that
take that canonical output and xlate it to whatever format
the end-user wants.

> On Nov 30, 2016, at 1:38 PM, Nick Kew  wrote:
> 
> On Wed, 2016-11-30 at 12:54 -0500, Jim Jagielski wrote:
>> I'm thinking about adding JSON support to mod_status...
>> the "plain" version output really stinks and lacks parity
>> w/ the info we provide via HTML, and it would be nice
>> to produce a really easily parseable format.
>> 
>> Thoughts...?
> 
> Ideally that should be the job of an output filter.
> If mod_status were to generate a single standard-ish
> format, like JSON or XML, a general-purpose filter
> could generate other formats.
> 
> -- 
> Nick Kew
> 

Re: JSON for mod_status

[William A Rowe Jr]

On Wed, Nov 30, 2016 at 12:38 PM, Nick Kew  wrote:

> On Wed, 2016-11-30 at 12:54 -0500, Jim Jagielski wrote:
> > I'm thinking about adding JSON support to mod_status...
> > the "plain" version output really stinks and lacks parity
> > w/ the info we provide via HTML, and it would be nice
> > to produce a really easily parseable format.
> >
> > Thoughts...?
>
> Ideally that should be the job of an output filter.
> If mod_status were to generate a single standard-ish
> format, like JSON or XML, a general-purpose filter
> could generate other formats.
>

The underlying flaw is our extensible status page output.

There's no way to anticipate the "right way" to map json
tables to presentation-level results. And modules may
express multiple tables. See the ssl session cache for
one good example, a simply key-value table of some of
the current metrics, and a longer table for data.

So there either needs to be an extensible data engine as
Aaron created, with each status module choosing how to
represent it's component data, or mod_status needs two
custom hooks, one chain for presentation data and one
chain for json data. Existing mod_status extentions in
modules only register the presentation hook, so they
would not be invoked on a json query.

mod_bmx avoids this quandary by placing a different
generator for json queries complete with filtering.

Based on the interest expressed on this thread, I'm
happy to put mod_bmx up for a vote tomorrow after
letting this thread run for a bit.

Re: JSON for mod_status

[William A Rowe Jr]

On Wed, Nov 30, 2016 at 11:54 AM, Jim Jagielski  wrote:

> I'm thinking about adding JSON support to mod_status...
> the "plain" version output really stinks and lacks parity
> w/ the info we provide via HTML, and it would be nice
> to produce a really easily parseable format.
>
> Thoughts...?
>

It's right here...

https://github.com/hyperic/mod_bmx/blob/master/README-BMX.txt

I'd brought this to the PMC's attention to see if there were sufficient
maintainers and interested parties. In short, the original author was
our own Bannert.

In short, Pivotal is offering this to the HTTPD project under AL 2.0
(as it's already licensed).

Right now, jfclere and I are the only two really looking at the code,
although it has many commercial adopters (the preferred httpd
introspection agent of the Hyperic monitoring software).

What bannert coded was more than converting status -> json.
He build a bean framework that is extensible to any other httpd
module. It seems like that bean framework might even be a
candidate to move into the httpd core with 2.next?

As far as extentions, his most useful was the mod_bmx_vhost
plug in. Taking it to the next step, that module should be able
to shoot out a nice presentation table for mod_status users,
while folks inspecting the bmx json bean interface already get
these vhost-specific breakdowns.

Re: JSON for mod_status

[Nick Kew]

On Wed, 2016-11-30 at 12:54 -0500, Jim Jagielski wrote:
> I'm thinking about adding JSON support to mod_status...
> the "plain" version output really stinks and lacks parity
> w/ the info we provide via HTML, and it would be nice
> to produce a really easily parseable format.
> 
> Thoughts...?

Ideally that should be the job of an output filter.
If mod_status were to generate a single standard-ish
format, like JSON or XML, a general-purpose filter
could generate other formats.

-- 
Nick Kew

Re: JSON for mod_status

[Kurt Newman]

+1

> On Nov 30, 2016, at 11:54 AM, Jim Jagielski  wrote:
> 
> I'm thinking about adding JSON support to mod_status...
> the "plain" version output really stinks and lacks parity
> w/ the info we provide via HTML, and it would be nice
> to produce a really easily parseable format.
> 
> Thoughts...?



smime.p7s
Description: S/MIME cryptographic signature

Re: JSON for mod_status

[Jacob Perkins]

Howdy,

I saw your tweet on that this morning, and I think it would be a useful 
addition to have mod_status output in a machine readable format for easy 
interpolation and munging.

+1

—
Jacob Perkins
Product Owner
cPanel Inc.

jacob.perk...@cpanel.net 
Office:  713-529-0800 x 4046
Cell:  713-560-8655

> On Nov 30, 2016, at 11:54 AM, Jim Jagielski  wrote:
> 
> I'm thinking about adding JSON support to mod_status...
> the "plain" version output really stinks and lacks parity
> w/ the info we provide via HTML, and it would be nice
> to produce a really easily parseable format.
> 
> Thoughts...?

Re: JSON for mod_status

[Daniel Gruno]

On 11/30/2016 07:20 PM, Jordan Gigov wrote:
> I think a better approach than "?json=true" would be to to respect the
> "Accept" header values of "application/json" and "text/json" if they
> have the higher priority.

Or both? Practically speaking, showing JSON based on a query arg is
vastly easier for laymen to implement in their applications than setting
accept headers, AND you can query and get JSON in your browser also.

> XML output should also be an option, if it will indeed serve as an API
> end-point.

Meh, I honestly have not used XML in a decade, so I'm +/-0 on that one.
If someone wants to do it, fine, but .. :p

> 
> On 30 November 2016 at 20:08, Luca Toscano  wrote:
>>
>>
>> 2016-11-30 19:03 GMT+01:00 Daniel Gruno :
>>>
>>> On 11/30/2016 06:54 PM, Jim Jagielski wrote:
 I'm thinking about adding JSON support to mod_status...
 the "plain" version output really stinks and lacks parity
 w/ the info we provide via HTML, and it would be nice
 to produce a really easily parseable format.

 Thoughts...?
>>
>>
>> +1, this will simplify a lot things like metrics collectors polling httpd
>> regularly (that don't need a nice html format).
>>
>>> +1, a ?json=true or some such to the query would be helpful.
>>> so you could do
>>> /server-status?json=true
>>> or
>>> /server-status?auto=true
>>>
>>
>> +1, I like the idea!
>>
>> Thanks!
>>
>> Luca
>>

Re: JSON for mod_status

[Jordan Gigov]

I think a better approach than "?json=true" would be to to respect the
"Accept" header values of "application/json" and "text/json" if they
have the higher priority.
XML output should also be an option, if it will indeed serve as an API
end-point.

On 30 November 2016 at 20:08, Luca Toscano  wrote:
>
>
> 2016-11-30 19:03 GMT+01:00 Daniel Gruno :
>>
>> On 11/30/2016 06:54 PM, Jim Jagielski wrote:
>> > I'm thinking about adding JSON support to mod_status...
>> > the "plain" version output really stinks and lacks parity
>> > w/ the info we provide via HTML, and it would be nice
>> > to produce a really easily parseable format.
>> >
>> > Thoughts...?
>
>
> +1, this will simplify a lot things like metrics collectors polling httpd
> regularly (that don't need a nice html format).
>
>> +1, a ?json=true or some such to the query would be helpful.
>> so you could do
>> /server-status?json=true
>> or
>> /server-status?auto=true
>>
>
> +1, I like the idea!
>
> Thanks!
>
> Luca
>

Re: JSON for mod_status

[Luca Toscano]

2016-11-30 19:03 GMT+01:00 Daniel Gruno :

> On 11/30/2016 06:54 PM, Jim Jagielski wrote:
> > I'm thinking about adding JSON support to mod_status...
> > the "plain" version output really stinks and lacks parity
> > w/ the info we provide via HTML, and it would be nice
> > to produce a really easily parseable format.
> >
> > Thoughts...?
>

+1, this will simplify a lot things like metrics collectors polling httpd
regularly (that don't need a nice html format).

+1, a ?json=true or some such to the query would be helpful.
> so you could do
> /server-status?json=true
> or
> /server-status?auto=true
>
>
+1, I like the idea!

Thanks!

Luca

Re: JSON for mod_status

[Daniel Gruno]

On 11/30/2016 06:54 PM, Jim Jagielski wrote:
> I'm thinking about adding JSON support to mod_status...
> the "plain" version output really stinks and lacks parity
> w/ the info we provide via HTML, and it would be nice
> to produce a really easily parseable format.
> 
> Thoughts...?
> 
+1, a ?json=true or some such to the query would be helpful.
so you could do
/server-status?json=true
or
/server-status?auto=true

etc.