Hi Rafel, this is really great stuff. It would help me to get forward with my wifi controller. Could it be possible to subsribe to multiple sources to limit the connections to ubus? 2 SSIDs with 2.4 ad 5GHz would me 4 concurrent channels if I understand right.
Kind regards, André Am 31.07.20 um 06:49 schrieb Rafał Miłecki: > From: Rafał Miłecki <ra...@milecki.pl> > > Initial uhttpd ubus API was fully based on JSON-RPC. That restricted it > from supporting ubus notifications that don't fit its model. > > Notifications require protocol that allows server to send data without > being polled. There are two candidates for that: > 1. Server-sent events > 2. WebSocket > > The later one is overcomplex for this simple task so ideally uhttps ubus > should support text-based server-sent events. It's not possible with > JSON-RPC without violating it. Specification requires server to reply > with Response object. Replying with text/event-stream is not allowed. > > All above led to designing new API that: > 1. Uses GET and POST requests > 2. Makes use of RESTful URLs > 3. Uses JSON-RPC in cleaner form and only for calling ubus methods > > This new API allows: > 1. Listing all ubus objects and their methods using GET <prefix>/list > 2. Listing object methods using GET <prefix>/list/<path> > 3. Listening to object notifications with GET <prefix>/subscribe/<path> > 4. Calling ubus methods using POST <prefix>/call/<path> > > JSON-RPC custom protocol was also simplified to: > 1. Use "method" member for ubus object method name > It was possible thanks to using RESTful URLs. Previously "method" > had to be "list" or "call". > 2. Reply with Error object on ubus method call error > This simplified "result" member format as it doesn't need to contain > ubus result code anymore. > > This patch doesn't break or change the old API. The biggest downside of > the new API is no support for batch requests. It's cost of using RESTful > URLs. It should not matter much as uhttpd supports keep alive. > > Example usages: > > 1. Getting all objects and their methods: > $ curl http://192.168.1.1/ubus/list > { > "dhcp": { > "ipv4leases": { > > }, > "ipv6leases": { > > } > }, > "log": { > "read": { > "lines": "number", > "stream": "boolean", > "oneshot": "boolean" > }, > "write": { > "event": "string" > } > } > } > > 2. Getting object methods: > $ curl http://192.168.1.1/ubus/list/log > { > "read": { > "lines": "number", > "stream": "boolean", > "oneshot": "boolean" > }, > "write": { > "event": "string" > } > } > > 3. Subscribing to notifications: > $ curl http://192.168.1.1/ubus/subscribe/foo > event: status > data: {"count":5} > > 4. Calling ubus object method: > $ curl -d '{ > "jsonrpc": "2.0", > "id": 1, > "method": "login", > "params": {"username": "root", "password": "password" } > }' http://192.168.1.1/ubus/call/session > { > "jsonrpc": "2.0", > "id": 1, > "result": { > "ubus_rpc_session": "01234567890123456789012345678901", > (...) > } > } > > $ curl -H 'Authorization: Bearer 01234567890123456789012345678901' -d '{ > "jsonrpc": "2.0", > "id": 1, > "method": "write", > "params": {"event": "Hello world" } > }' http://192.168.1.1/ubus/call/log > { > "jsonrpc": "2.0", > "id": 1, > "result": null > } > > Signed-off-by: Rafał Miłecki <ra...@milecki.pl> > --- > V2: Use "Authorization" with Bearer for rpcd session id / token > Treat missing session id as UH_UBUS_DEFAULT_SID > Fix "result" format (was: "result":{{"foo":"bar"}}) > --- > main.c | 8 +- > ubus.c | 326 +++++++++++++++++++++++++++++++++++++++++++++++++++---- > uhttpd.h | 5 + > 3 files changed, 318 insertions(+), 21 deletions(-) > > diff --git a/main.c b/main.c > index 26e74ec..73e3d42 100644 > --- a/main.c > +++ b/main.c > @@ -159,6 +159,7 @@ static int usage(const char *name) > " -U file Override ubus socket path\n" > " -a Do not authenticate JSON-RPC requests > against UBUS session api\n" > " -X Enable CORS HTTP headers on JSON-RPC > api\n" > + " -e Events subscription reconnection time > (retry value)\n" > #endif > " -x string URL prefix for CGI handler, default is > '/cgi-bin'\n" > " -y alias[=path] URL alias handle\n" > @@ -262,7 +263,7 @@ int main(int argc, char **argv) > init_defaults_pre(); > signal(SIGPIPE, SIG_IGN); > > - while ((ch = getopt(argc, argv, > "A:aC:c:Dd:E:fh:H:I:i:K:k:L:l:m:N:n:P:p:qRr:Ss:T:t:U:u:Xx:y:")) != -1) { > + while ((ch = getopt(argc, argv, > "A:aC:c:Dd:E:e:fh:H:I:i:K:k:L:l:m:N:n:P:p:qRr:Ss:T:t:U:u:Xx:y:")) != -1) { > switch(ch) { > #ifdef HAVE_TLS > case 'C': > @@ -490,11 +491,16 @@ int main(int argc, char **argv) > case 'X': > conf.ubus_cors = 1; > break; > + > + case 'e': > + conf.events_retry = atoi(optarg); > + break; > #else > case 'a': > case 'u': > case 'U': > case 'X': > + case 'e': > fprintf(stderr, "uhttpd: UBUS support not compiled, " > "ignoring -%c\n", ch); > break; > diff --git a/ubus.c b/ubus.c > index c22e07a..fd907db 100644 > --- a/ubus.c > +++ b/ubus.c > @@ -73,6 +73,7 @@ struct rpc_data { > > struct list_data { > bool verbose; > + bool add_object; > struct blob_buf *buf; > }; > > @@ -154,14 +155,14 @@ static void uh_ubus_add_cors_headers(struct client *cl) > ustream_printf(cl->us, "Access-Control-Allow-Credentials: true\r\n"); > } > > -static void uh_ubus_send_header(struct client *cl) > +static void uh_ubus_send_header(struct client *cl, int code, const char > *summary, const char *content_type) > { > - ops->http_header(cl, 200, "OK"); > + ops->http_header(cl, code, summary); > > if (conf.ubus_cors) > uh_ubus_add_cors_headers(cl); > > - ustream_printf(cl->us, "Content-Type: application/json\r\n"); > + ustream_printf(cl->us, "Content-Type: %s\r\n", content_type); > > if (cl->request.method == UH_HTTP_MSG_OPTIONS) > ustream_printf(cl->us, "Content-Length: 0\r\n"); > @@ -217,12 +218,165 @@ static void uh_ubus_json_rpc_error(struct client *cl, > enum rpc_error type) > uh_ubus_send_response(cl); > } > > +static void uh_ubus_error(struct client *cl, int code, const char *message) > +{ > + blobmsg_add_u32(&buf, "code", code); > + blobmsg_add_string(&buf, "message", message); > + uh_ubus_send_response(cl); > +} > + > +static void uh_ubus_posix_error(struct client *cl, int err) > +{ > + uh_ubus_error(cl, -err, strerror(err)); > +} > + > +static void uh_ubus_ubus_error(struct client *cl, int err) > +{ > + uh_ubus_error(cl, err, ubus_strerror(err)); > +} > + > +/* GET requests handling */ > + > +static void uh_ubus_list_cb(struct ubus_context *ctx, struct > ubus_object_data *obj, void *priv); > + > +static void uh_ubus_handle_get_list(struct client *cl, const char *path) > +{ > + static struct blob_buf tmp; > + struct list_data data = { .verbose = true, .add_object = !path, .buf = > &tmp}; > + struct blob_attr *cur; > + int rem; > + int err; > + > + blob_buf_init(&tmp, 0); > + > + err = ubus_lookup(ctx, path, uh_ubus_list_cb, &data); > + if (err) { > + uh_ubus_send_header(cl, 500, "Ubus Protocol Error", > "application/json"); > + uh_ubus_ubus_error(cl, err); > + return; > + } > + > + uh_ubus_send_header(cl, 200, "OK", "application/json"); > + blob_for_each_attr(cur, tmp.head, rem) > + blobmsg_add_blob(&buf, cur); > + uh_ubus_send_response(cl); > +} > + > +static int uh_ubus_subscription_notification_cb(struct ubus_context *ctx, > + struct ubus_object *obj, > + struct ubus_request_data *req, > + const char *method, > + struct blob_attr *msg) > +{ > + struct ubus_subscriber *s; > + struct dispatch_ubus *du; > + struct client *cl; > + char *json; > + > + s = container_of(obj, struct ubus_subscriber, obj); > + du = container_of(s, struct dispatch_ubus, sub); > + cl = container_of(du, struct client, dispatch.ubus); > + > + json = blobmsg_format_json(msg, true); > + if (json) { > + ops->chunk_printf(cl, "event: %s\ndata: %s\n\n", method, json); > + free(json); > + } > + > + return 0; > +} > + > +static void uh_ubus_subscription_notification_remove_cb(struct ubus_context > *ctx, struct ubus_subscriber *s, uint32_t id) > +{ > + struct dispatch_ubus *du; > + struct client *cl; > + > + du = container_of(s, struct dispatch_ubus, sub); > + cl = container_of(du, struct client, dispatch.ubus); > + > + ops->request_done(cl); > +} > + > +static void uh_ubus_handle_get_subscribe(struct client *cl, const char *sid, > const char *path) > +{ > + struct dispatch_ubus *du = &cl->dispatch.ubus; > + uint32_t id; > + int err; > + > + /* TODO: add ACL support */ > + if (!conf.ubus_noauth) { > + uh_ubus_send_header(cl, 200, "OK", "application/json"); > + uh_ubus_posix_error(cl, EACCES); > + return; > + } > + > + du->sub.cb = uh_ubus_subscription_notification_cb; > + du->sub.remove_cb = uh_ubus_subscription_notification_remove_cb; > + > + uh_client_ref(cl); > + > + err = ubus_register_subscriber(ctx, &du->sub); > + if (err) > + goto err_unref; > + > + err = ubus_lookup_id(ctx, path, &id); > + if (err) > + goto err_unregister; > + > + err = ubus_subscribe(ctx, &du->sub, id); > + if (err) > + goto err_unregister; > + > + uh_ubus_send_header(cl, 200, "OK", "text/event-stream"); > + > + if (conf.events_retry) > + ops->chunk_printf(cl, "retry: %d\n", conf.events_retry); > + > + return; > + > +err_unregister: > + ubus_unregister_subscriber(ctx, &du->sub); > +err_unref: > + uh_client_unref(cl); > + if (err) { > + uh_ubus_send_header(cl, 200, "OK", "application/json"); > + uh_ubus_ubus_error(cl, err); > + } > +} > + > +static void uh_ubus_handle_get(struct client *cl) > +{ > + struct dispatch_ubus *du = &cl->dispatch.ubus; > + const char *url = du->url; > + > + url += strlen(conf.ubus_prefix); > + > + if (!strcmp(url, "/list") || !strncmp(url, "/list/", strlen("/list/"))) > { > + url += strlen("/list"); > + > + uh_ubus_handle_get_list(cl, *url ? url + 1 : NULL); > + } else if (!strncmp(url, "/subscribe/", strlen("/subscribe/"))) { > + url += strlen("/subscribe"); > + > + uh_ubus_handle_get_subscribe(cl, NULL, url + 1); > + } else { > + ops->http_header(cl, 404, "Not Found"); > + ustream_printf(cl->us, "\r\n"); > + ops->request_done(cl); > + } > +} > + > +/* POST requests handling */ > + > static void > uh_ubus_request_data_cb(struct ubus_request *req, int type, struct blob_attr > *msg) > { > struct dispatch_ubus *du = container_of(req, struct dispatch_ubus, req); > + struct blob_attr *cur; > + int len; > > - blobmsg_add_field(&du->buf, BLOBMSG_TYPE_TABLE, "", blob_data(msg), > blob_len(msg)); > + blob_for_each_attr(cur, msg, len) > + blobmsg_add_blob(&du->buf, cur); > } > > static void > @@ -235,13 +389,46 @@ uh_ubus_request_cb(struct ubus_request *req, int ret) > int rem; > > uloop_timeout_cancel(&du->timeout); > - uh_ubus_init_json_rpc_response(cl); > - r = blobmsg_open_array(&buf, "result"); > - blobmsg_add_u32(&buf, "", ret); > - blob_for_each_attr(cur, du->buf.head, rem) > - blobmsg_add_blob(&buf, cur); > - blobmsg_close_array(&buf, r); > - uh_ubus_send_response(cl); > + > + /* Legacy format always uses "result" array - even for errors and empty > + * results. */ > + if (du->legacy) { > + void *c; > + > + uh_ubus_init_json_rpc_response(cl); > + r = blobmsg_open_array(&buf, "result"); > + blobmsg_add_u32(&buf, "", ret); > + c = blobmsg_open_table(&buf, NULL); > + blob_for_each_attr(cur, du->buf.head, rem) > + blobmsg_add_blob(&buf, cur); > + blobmsg_close_table(&buf, c); > + blobmsg_close_array(&buf, r); > + uh_ubus_send_response(cl); > + return; > + } > + > + if (ret) { > + void *c; > + > + uh_ubus_init_json_rpc_response(cl); > + c = blobmsg_open_table(&buf, "error"); > + blobmsg_add_u32(&buf, "code", ret); > + blobmsg_add_string(&buf, "message", ubus_strerror(ret)); > + blobmsg_close_table(&buf, c); > + uh_ubus_send_response(cl); > + } else { > + uh_ubus_init_json_rpc_response(cl); > + if (blob_len(du->buf.head)) { > + r = blobmsg_open_table(&buf, "result"); > + blob_for_each_attr(cur, du->buf.head, rem) > + blobmsg_add_blob(&buf, cur); > + blobmsg_close_table(&buf, r); > + } else { > + blobmsg_add_field(&buf, BLOBMSG_TYPE_UNSPEC, "result", > NULL, 0); > + } > + uh_ubus_send_response(cl); > + } > + > } > > static void > @@ -282,7 +469,7 @@ static void uh_ubus_request_free(struct client *cl) > > static void uh_ubus_single_error(struct client *cl, enum rpc_error type) > { > - uh_ubus_send_header(cl); > + uh_ubus_send_header(cl, 200, "OK", "application/json"); > uh_ubus_json_rpc_error(cl, type); > ops->request_done(cl); > } > @@ -335,7 +522,8 @@ static void uh_ubus_list_cb(struct ubus_context *ctx, > struct ubus_object_data *o > if (!obj->signature) > return; > > - o = blobmsg_open_table(data->buf, obj->path); > + if (data->add_object) > + o = blobmsg_open_table(data->buf, obj->path); > blob_for_each_attr(sig, obj->signature, rem) { > t = blobmsg_open_table(data->buf, blobmsg_name(sig)); > rem2 = blobmsg_data_len(sig); > @@ -366,13 +554,14 @@ static void uh_ubus_list_cb(struct ubus_context *ctx, > struct ubus_object_data *o > } > blobmsg_close_table(data->buf, t); > } > - blobmsg_close_table(data->buf, o); > + if (data->add_object) > + blobmsg_close_table(data->buf, o); > } > > static void uh_ubus_send_list(struct client *cl, struct blob_attr *params) > { > struct blob_attr *cur, *dup; > - struct list_data data = { .buf = &cl->dispatch.ubus.buf, .verbose = > false }; > + struct list_data data = { .buf = &cl->dispatch.ubus.buf, .verbose = > false, .add_object = true }; > void *r; > int rem; > > @@ -471,7 +660,7 @@ static void uh_ubus_init_batch(struct client *cl) > struct dispatch_ubus *du = &cl->dispatch.ubus; > > du->array = true; > - uh_ubus_send_header(cl); > + uh_ubus_send_header(cl, 200, "OK", "application/json"); > ops->chunk_printf(cl, "["); > } > > @@ -594,7 +783,7 @@ static void uh_ubus_data_done(struct client *cl) > > switch (obj ? json_object_get_type(obj) : json_type_null) { > case json_type_object: > - uh_ubus_send_header(cl); > + uh_ubus_send_header(cl, 200, "OK", "application/json"); > return uh_ubus_handle_request_object(cl, obj); > case json_type_array: > uh_ubus_init_batch(cl); > @@ -604,6 +793,96 @@ static void uh_ubus_data_done(struct client *cl) > } > } > > +static void uh_ubus_call(struct client *cl, const char *path, const char > *sid) > +{ > + struct dispatch_ubus *du = &cl->dispatch.ubus; > + struct json_object *obj = du->jsobj; > + struct rpc_data data = {}; > + enum rpc_error err = ERROR_PARSE; > + static struct blob_buf req; > + > + uh_client_ref(cl); > + > + if (!obj || json_object_get_type(obj) != json_type_object) > + goto error; > + > + uh_ubus_send_header(cl, 200, "OK", "application/json"); > + > + du->jsobj_cur = obj; > + blob_buf_init(&req, 0); > + if (!blobmsg_add_object(&req, obj)) > + goto error; > + > + if (!parse_json_rpc(&data, req.head)) > + goto error; > + > + du->func = data.method; > + if (ubus_lookup_id(ctx, path, &du->obj)) { > + err = ERROR_OBJECT; > + goto error; > + } > + > + if (!conf.ubus_noauth && !uh_ubus_allowed(sid, path, data.method)) { > + err = ERROR_ACCESS; > + goto error; > + } > + > + uh_ubus_send_request(cl, sid, data.params); > + goto out; > + > +error: > + uh_ubus_json_rpc_error(cl, err); > +out: > + if (data.params) > + free(data.params); > + > + uh_client_unref(cl); > +} > + > +enum ubus_hdr { > + HDR_AUTHORIZATION, > + __HDR_UBUS_MAX > +}; > + > +static void uh_ubus_handle_post(struct client *cl) > +{ > + static const struct blobmsg_policy hdr_policy[__HDR_UBUS_MAX] = { > + [HDR_AUTHORIZATION] = { "authorization", BLOBMSG_TYPE_STRING }, > + }; > + struct dispatch_ubus *du = &cl->dispatch.ubus; > + struct blob_attr *tb[__HDR_UBUS_MAX]; > + const char *url = du->url; > + const char *auth; > + > + if (!strcmp(url, conf.ubus_prefix)) { > + du->legacy = true; > + uh_ubus_data_done(cl); > + return; > + } > + > + blobmsg_parse(hdr_policy, __HDR_UBUS_MAX, tb, blob_data(cl->hdr.head), > blob_len(cl->hdr.head)); > + > + auth = UH_UBUS_DEFAULT_SID; > + if (tb[HDR_AUTHORIZATION]) { > + const char *tmp = blobmsg_get_string(tb[HDR_AUTHORIZATION]); > + > + if (!strncasecmp(tmp, "Bearer ", 7)) > + auth = tmp + 7; > + } > + > + url += strlen(conf.ubus_prefix); > + > + if (!strncmp(url, "/call/", strlen("/call/"))) { > + url += strlen("/call/"); > + > + uh_ubus_call(cl, url, auth); > + } else { > + ops->http_header(cl, 404, "Not Found"); > + ustream_printf(cl->us, "\r\n"); > + ops->request_done(cl); > + } > +} > + > static int uh_ubus_data_send(struct client *cl, const char *data, int len) > { > struct dispatch_ubus *du = &cl->dispatch.ubus; > @@ -626,21 +905,28 @@ error: > static void uh_ubus_handle_request(struct client *cl, char *url, struct > path_info *pi) > { > struct dispatch *d = &cl->dispatch; > + struct dispatch_ubus *du = &d->ubus; > > blob_buf_init(&buf, 0); > > + du->url = url; > + du->legacy = false; > + > switch (cl->request.method) > { > + case UH_HTTP_MSG_GET: > + uh_ubus_handle_get(cl); > + break; > case UH_HTTP_MSG_POST: > d->data_send = uh_ubus_data_send; > - d->data_done = uh_ubus_data_done; > + d->data_done = uh_ubus_handle_post; > d->close_fds = uh_ubus_close_fds; > d->free = uh_ubus_request_free; > - d->ubus.jstok = json_tokener_new(); > + du->jstok = json_tokener_new(); > break; > > case UH_HTTP_MSG_OPTIONS: > - uh_ubus_send_header(cl); > + uh_ubus_send_header(cl, 200, "OK", "application/json"); > ops->request_done(cl); > break; > > diff --git a/uhttpd.h b/uhttpd.h > index f77718e..75dd747 100644 > --- a/uhttpd.h > +++ b/uhttpd.h > @@ -82,6 +82,7 @@ struct config { > int ubus_noauth; > int ubus_cors; > int cgi_prefix_len; > + int events_retry; > struct list_head cgi_alias; > struct list_head lua_prefix; > }; > @@ -209,6 +210,7 @@ struct dispatch_ubus { > struct json_tokener *jstok; > struct json_object *jsobj; > struct json_object *jsobj_cur; > + const char *url; > int post_len; > > uint32_t obj; > @@ -218,6 +220,9 @@ struct dispatch_ubus { > bool req_pending; > bool array; > int array_idx; > + bool legacy; /* Got legacy request => use legacy reply */ > + > + struct ubus_subscriber sub; > }; > #endif > > -- Mit freundlichen Grüßen André Valentin Systemadministration - Projektkoordination -- MarcanT AG, Herforder Straße 163a, D - 33609 Bielefeld Fon: +49 (521) 95945-0 | Fax: +49 (521) 95945-18 URL: http://www.marcant.net <http://www.marcant.net/> | http://www.global-m2m.com <http://www.global-m2m.com/> Internet * Netzwerk * Mobile Daten Vorstand: Thorsten Hojas (Vorsitzender) Marc-Henrik Delker Dr. Anja-Christina Padberg Handelsregister: AG Bielefeld, HRB 42260 USt-ID Nr.: DE 190203238 ___________________________________________________________ Ausserhalb unserer Geschäftszeiten (Montag bis Freitag von 8:30 Uhr bis 17:30 Uhr, ausgenommen gesetzliche Feiertage in NRW) stehen wir Ihnen gemäß Ihrer jeweiligen Service-Level-Agreements unter der Ihnen mitgeteilten Telefonnummer für Störungen und Notfälle zur Verfügung. Sie können natürlich auch gerne jederzeit unter supp...@marcant.net ein Ticket eröffnen, welches am nächsten Arbeitstag bearbeitet wird.
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
