RE: JSON for mod_status

[PKU . 孙斌]

Since JSON is now so important to the Web client-end world, it seems that Httpd 
would be better to support JSON data i/o filtering with a JSON parser 
integrated, or put it into the Libapr?

Bing

---
From: dev-return-87644-bswen=pku.edu...@httpd.apache.org 
[mailto:dev-return-87644-bswen=pku.edu...@httpd.apache.org] 代表 Houser, Rick

Personally, I'd rather have XML, but it may make sense to support multiple 
machine readable formats down the line.  As such, using a new parameter for 
every possibility gets messy.  If this gets implemented as a get parameter, how 
about making one parameter with multiple potential values down the line?  It's 
not like they would all have to be implemented up-front.

?type=json
?type=xml
?type=js
?type=csv
Etc.

As to accept headers, that's a pain in the butt if someone just wants to grab a 
copy of the data for debugging a custom parser or something.  So much easier to 
just wget BLAH than to mess with the accept headers on any utility we want to 
fetch with.


Rick Houser
Web Administration

> -Original Message-
> From: Jordan Gigov [mailto:colad...@gmail.com]
> Sent: Wednesday, November 30, 2016 13:20
> To: dev@httpd.apache.org
> Subject: Re: JSON for mod_status
> 
> I think a better approach than "?json=true" would be to to respect the
> "Accept" header values of "application/json" and "text/json" if they
> have the higher priority.
> XML output should also be an option, if it will indeed serve as an API
> end-point.
> 
> On 30 November 2016 at 20:08, Luca Toscano 
> wrote:
> >
> >
> > 2016-11-30 19:03 GMT+01:00 Daniel Gruno :
> >>
> >> On 11/30/2016 06:54 PM, Jim Jagielski wrote:
> >> > I'm thinking about adding JSON support to mod_status...
> >> > the "plain" version output really stinks and lacks parity
> >> > w/ the info we provide via HTML, and it would be nice
> >> > to produce a really easily parseable format.
> >> >
> >> > Thoughts...?
> >
> >
> > +1, this will simplify a lot things like metrics collectors polling httpd
> > regularly (that don't need a nice html format).
> >
> >> +1, a ?json=true or some such to the query would be helpful.
> >> so you could do
> >> /server-status?json=true
> >> or
> >> /server-status?auto=true
> >>
> >
> > +1, I like the idea!
> >
> > Thanks!
> >
> > Luca
> >

Re: JSON for mod_status

[Jim Jagielski]

> On Dec 1, 2016, at 2:16 PM, William A Rowe Jr  wrote:
> 
> 
> Note that mod_bmx_status entries focus on what most management
> frameworks are looking for, so thus far, it doesn't deliver an entire
> dataset per connection or worker thread. It certainly could, of course.

Nor does it expose anything from any other beans created
by any other module.

Can it?

If so, exactly how.

> 
> Again, it simply iterates the entire contents of each bean presented
> to it's own print_bean_fn callback. If you have a look at;
> https://github.com/hyperic/mod_bmx/blob/master/modules/bmx/mod_bmx.h
> that should give you a somewhat higher altitude perspective on Aaron's
> API design.

So mod_bmx_status has to query mod_bmx for "all beans" ??

Re: Time for 2.4.24!

[William A Rowe Jr]

On Thu, Dec 1, 2016 at 9:06 AM, Eric Covener  wrote:

> Trying to look at the event stuff and the strict stuff.
>
> If any more experienced event folks are lurking, help on the review
> would be great (And thanks to sf!)
>

I found the PR itself to be full of useful data for review (including some
reporters' test results against the proposed patches).

Re: JSON for mod_status

[William A Rowe Jr]

On Thu, Dec 1, 2016 at 12:33 PM, Jim Jagielski  wrote:

>
> > On Dec 1, 2016, at 12:53 PM, William A Rowe Jr 
> wrote:
> >
> > Finally the query fn in mod_bmx_status performs the callback indicated
> > through its invocation to unspool the data in presentation format, which
> > lives back in mod_bmx and behaves identically for every bmx extension.
> > Adding the properties by type takes all of the onus off of each extension
> > to know how to represent them once stored. E.g. our query_hook function
> > can be as simple as;
> > /* create the bean */
> > bmx_bean_create(_status_bean, bmx_status_objectname, r->pool);
> >
> > bmx_bean_prop_add(bmx_status_bean,
> > bmx_property_string_create("ServerName",
> >ap_get_server_name(r),
> >r->pool));print_bean_fn(r,
> bmx_status_bean);
> >
> >
>
> OK, this is the point I have the question about. From what
> I can see, mod_bmx_status creates bmx_status_bean, adds stuff
> to it, and then prints it out.
>
> But how does it get access to, for example, the beans created and
> populated in mod_bmx_vhost? I understand that if you KNOW what
> beans you are looking for, you can query them via mod_bmx, but
> how do you know what beans have been added.
>

The beans are individually reflected to the print_bean_fn by each of
the consumers who registered against the query_hook.

mod_bmx then iterates the name of the bean and the property names
to filter out what is wanted. If *everything* is wanted, simply don't give
any query args to the bmx-handler request.


> Right now, for example, we have a simple hook that mod_socache_redis
> (for example) hooks into and once mod_status is done, it loops
> through all registered hooks. But I don't see any similar type of
> functionality in mod_bmx_status which allows mod_bmx_status to
> reproduce the full status information that mod_status currently
> does.
>

Note that mod_bmx_status entries focus on what most management
frameworks are looking for, so thus far, it doesn't deliver an entire
dataset per connection or worker thread. It certainly could, of course.

One thing not implemented, json (or even bean) arrays, which might
become very useful for tabular data from status, balancer, etc.

So yeah, other modules create their beans. Fine. But how
> do those beans get "registered" in mod_bmx_status so that they
> can actually be displayed?


Again, it simply iterates the entire contents of each bean presented
to it's own print_bean_fn callback. If you have a look at;
https://github.com/hyperic/mod_bmx/blob/master/modules/bmx/mod_bmx.h
that should give you a somewhat higher altitude perspective on Aaron's
API design.

More background;
https://github.com/hyperic/mod_bmx/blob/master/README-BMX.txt

Re: JSON for mod_status

[Jim Jagielski]

> On Dec 1, 2016, at 12:53 PM, William A Rowe Jr  wrote:
> 
> 
> Finally the query fn in mod_bmx_status performs the callback indicated
> through its invocation to unspool the data in presentation format, which
> lives back in mod_bmx and behaves identically for every bmx extension.
> Adding the properties by type takes all of the onus off of each extension
> to know how to represent them once stored. E.g. our query_hook function
> can be as simple as;
> /* create the bean */
> bmx_bean_create(_status_bean, bmx_status_objectname, r->pool);
> 
> bmx_bean_prop_add(bmx_status_bean,
> bmx_property_string_create("ServerName",
>ap_get_server_name(r),
>r->pool));print_bean_fn(r, 
> bmx_status_bean);
> 
> 

OK, this is the point I have the question about. From what
I can see, mod_bmx_status creates bmx_status_bean, adds stuff
to it, and then prints it out.

But how does it get access to, for example, the beans created and
populated in mod_bmx_vhost? I understand that if you KNOW what
beans you are looking for, you can query them via mod_bmx, but
how do you know what beans have been added.

Right now, for example, we have a simple hook that mod_socache_redis
(for example) hooks into and once mod_status is done, it loops
through all registered hooks. But I don't see any similar type of
functionality in mod_bmx_status which allows mod_bmx_status to
reproduce the full status information that mod_status currently
does.

So yeah, other modules create their beans. Fine. But how
do those beans get "registered" in mod_bmx_status so that they
can actually be displayed?

Re: JSON for mod_status

[William A Rowe Jr]

On Thu, Dec 1, 2016 at 8:08 AM, Jim Jagielski  wrote:

> My question is how do the beans, for example, from mod_bmx_vhost
> get displayed via mod_bmx_status?
>
> I understand that one queries mod_bmx for specific beans, but at
> the end of the day people still want/need a comprehensize
> mod_status-like display. My expectation would be that mod_bmx_status
> would provide that.
>


mod_bmx is the hub module which handles the bmx-handler generator.
Configuration is as straightforward as compiling in or loading mod_bmx.so
and following a typical server-status handler config;


SetHandler bmx-handler
Require ip 127.0.0.1


What mod_bmx reports depends on who hooks into it. You can run it
by only loading mod_bmx_status, or only mod_bmx_vhost, or any
combination of multiple providers. An obvious new extension would
be mod_proxy_balancer pool load reporting.

mod_bmx_status hooks into this framework by setting a server-lifetime
bean indicating that ServerStatus is extended or not, and then hooks
into the mod_bmx query hook during the pre-config phase;

APR_OPTIONAL_HOOK(bmx, query_hook, bmx_status_query_hook, NULL, NULL,
  APR_HOOK_MIDDLE);

Our mod_bmx_status query_hook handler is a fairly typical 'handler',
of data instead of content;

static int bmx_status_query_hook(request_rec *r,
 const struct bmx_objectname *query,
 bmx_bean_print print_bean_fn)


We don't want to refresh our data if the query indicates that the request
to the bmx-handler excluded server-status data, so there's a short circuit
right off the bat. Provided some subset of server-status data is needed,
this function continues by populating the data for mod_bmx to present
(which it can further filter by item name without help from the extensions).

Finally the query fn in mod_bmx_status performs the callback indicated
through its invocation to unspool the data in presentation format, which
lives back in mod_bmx and behaves identically for every bmx extension.
Adding the properties by type takes all of the onus off of each extension
to know how to represent them once stored. E.g. our query_hook function
can be as simple as;

/* create the bean */
bmx_bean_create(_status_bean, bmx_status_objectname, r->pool);

bmx_bean_prop_add(bmx_status_bean,
bmx_property_string_create("ServerName",
   ap_get_server_name(r),
   r->pool));print_bean_fn(r,
bmx_status_bean);


For Stefan's and others concerns, it would probably be most useful
for Doug or Aaron to explain how they came to the bean datum model
for representing bmx's results.



.

Re: Time for 2.4.24!

[Eric Covener]

Trying to look at the event stuff and the strict stuff.

If any more experienced event folks are lurking, help on the review
would be great (And thanks to sf!)

On Thu, Dec 1, 2016 at 9:28 AM, Jim Jagielski  wrote:
> hello? hello? Anyone there? :)
>
>> On Nov 21, 2016, at 7:47 AM, Jim Jagielski  wrote:
>>
>> We have a few items in STATUS that, imo, should be tested, voted-on
>> and the committed to the httpd-2.4 branch in anticipation of a new
>> release SOON! I'd like to have a T the end of next week, if
>> possible so we can start off December (or be close to "starting
>> it off") with a new release for the community.
>>
>> I'll RM.
>



-- 
Eric Covener
cove...@gmail.com

Re: Time for 2.4.24!

[Stefan Eissing]

Don't look at me! Have my hands full with h2 stuff... ;-)

> Am 01.12.2016 um 15:28 schrieb Jim Jagielski :
> 
> hello? hello? Anyone there? :)
> 
>> On Nov 21, 2016, at 7:47 AM, Jim Jagielski  wrote:
>> 
>> We have a few items in STATUS that, imo, should be tested, voted-on
>> and the committed to the httpd-2.4 branch in anticipation of a new
>> release SOON! I'd like to have a T the end of next week, if
>> possible so we can start off December (or be close to "starting
>> it off") with a new release for the community.
>> 
>> I'll RM.
> 

Re: JSON for mod_status

[Jim Jagielski]

My question is how do the beans, for example, from mod_bmx_vhost
get displayed via mod_bmx_status?

I understand that one queries mod_bmx for specific beans, but at
the end of the day people still want/need a comprehensize
mod_status-like display. My expectation would be that mod_bmx_status
would provide that.

> On Nov 30, 2016, at 5:01 PM, William A Rowe Jr  wrote:
> 
> You are looking at one fold of monitoring data. See, for example, 
> mod_bmx_vhost which further extends many beans by hostname.
> 
> The response generator and framework lives in mod_bmx itself.
> 
> 
> On Nov 30, 2016 14:35, "Jim Jagielski"  wrote:
> One thing that I can't understand from an initial look
> is how the whole hook thing is. In mod_status, we have a hook
> that is run in mod_status and other modules use to supplement
> the mod_status output.
> 
> With mod_bmx it looks like instead whatever "chunk" of data
> you want to make available, you put into a bean, but mod_bmx_status
> uses a local bean with no provision to print out other
> beans as well.
> 
> How does one "extend" the info printed by mod_bmx_status
> via other beans added and populated by other modules?
> 
> 

Re: JSON for mod_status

[Stefan Eissing]

First: +1 to have machine-readable output from mod_status, maybe we'll even add 
test cases!

Second: -1 to have various hooks/sub-modules serialize JSON by themselves (or 
other formats). Instead, I'd rather have us chose a data/bean/binary json API 
with which to construct objects to pass around and have a standard set of 
serializers/parsers for JSON, XML, HTML, etc.

As to passing such data around, I've made very good experiences with wrapping 
response status/headers into their own meta bucket type for HTTP/2. That way 
the raw data gets passed around as buckets/through brigades and the main 
connection can serialize that to HTTP/2 frames.

-Stefan

> Am 30.11.2016 um 21:34 schrieb Jim Jagielski :
> 
> One thing that I can't understand from an initial look
> is how the whole hook thing is. In mod_status, we have a hook
> that is run in mod_status and other modules use to supplement
> the mod_status output.
> 
> With mod_bmx it looks like instead whatever "chunk" of data
> you want to make available, you put into a bean, but mod_bmx_status
> uses a local bean with no provision to print out other
> beans as well.
> 
> How does one "extend" the info printed by mod_bmx_status
> via other beans added and populated by other modules?
>