Hi Ilya, thanks for sharing your thoughts, always appreciated! ☺️ Please find comments below.
On 28.10.23 00:05, Ilya Maximets wrote: > On 10/27/23 23:51, Ilya Maximets wrote: >> On 10/26/23 13:44, Jakob Meng wrote: >>> On 25.10.23 11:37, [email protected] wrote: >>>> From: Jakob Meng <[email protected]> >>>> >>>> For monitoring systems such as Prometheus it would be beneficial if >>>> OVS and OVS-DPDK would expose statistics in a machine-readable format. > BTW, there is no such separate thing as OVS-DPDK, it's just OVS. With OVS-DPDK I wanted to highlight one of the potential use cases of this change. It came up in discussions and the OVS codebase. Will remove it, if it is frowned upon, no worry. > >>>> This patch introduces support for different output formats to ovs-xxx >>>> tools. They gain a global option '-f,--format' which allows users to >>>> request JSON instead of plain-text for humans. An example call >>>> implemented in a later patch is 'ovs-appctl --format json dpif/show'. >>>> >>>> For that it is necessary to change the JSON-RPC API lib/unixctl.* >>>> which ovs-xxx tools use to communicate with ovs-vswitchd across unix >>>> sockets. It now allows to transport the requested output format >>>> besides the command and its args. This change has been implemented in >>>> a backward compatible way. One can use an updated client >>>> (ovs-appctl/dpctl) with an old server (ovs-vswitchd) and vice versa. >>>> Of course, JSON output only works when both sides have been updated. >>>> >>>> Previously, the command was copied to the 'method' parameter in >>>> JSON-RPC while the args were copied to the 'params' parameter. Without >>>> any other means to transport parameters via JSON-RPC left, the meaning >>>> of 'method' and 'params' had to be changed: 'method' will now be >>>> 'execute/v1' when an output format other than 'text' is requested. In >>>> that case, the first parameter of the JSON array 'params' will now be >>>> the designated command, the second one the output format and the rest >>>> will be command args. >>> Ilya brought up the question why I changed the meaning of 'method' and >>> 'params' instead of adding the output format as an addition argument to the >>> command arguments in 'params'. The server side would then interpret and >>> filter out this argument before passing the remaining arguments to the >>> command callbacks. >>> >>> I decided against this approach because the code would get more involved, >>> in particular we would have to implement option/argument parsing inside >>> process_command() in lib/unixctl.c. (Besides, using getopt*() functions in >>> a safe way would be difficult in general because their global state.) >>> The current implementation is based purely on JSON objects/arrays which are >>> nicely supported by OVS with functions from lib/json.*. >> I'm not sure I got this point, but see below. >> >>> Ilya also voiced concerns about the limited extensibility of the proposed >>> API. To fix this, this patch series could be tweaked in the follow way: >>> >>> (1.) Instead of passing the output format as second entry in the JSON array >>> 'params', turn the second entry into a JSON object (shash). > It has to be an array, not an object. Parameters are positional. > Unless you want to change API for every existing command and give > each argument a unique name. > > And that will not help with supporting older servers, because they > expect an array. JSON-RPC 1.0 defines 'params' as a "Array of objects to pass as arguments to the method." [0]. Putting JSON objects into that array is valid. Please have a look at the source code changes in unixctl.c. Hopefully it helps with understanding my approach: Basically, I changed the meaning of the JSON-RPC API. Previously, there was a 1-1 mapping between command+args and JSON-RPC's method+params. Now, command+args are part of 'params'. [0] https://www.jsonrpc.org/specification_v1 (After reading though your mail completely, I am thinking about reevaluating this decision as explained below.) > >>> The output format would be one entry in this JSON object, e.g. called >>> 'format'. >>> >>> The server (process_command() from lib/unixctl.c) will parse the content of >>> this JSON object into known options. Clients would only add non-default >>> options to this JSON object to keep compatibility with older servers. >>> Options which are supported by the server but not transferred via this JSON >>> object would be initialized with default values to keep compatibility with >>> older clients. Unknown entries would cause the server to return an error. >> This kind of implements what I had in mind, but in a slightly different >> manner. How about something like this: >> >> * ovs-appctl has a global parameter --format passed to it, not part of >> the requested command paramaters. (I think this is implemented in the >> code, below, but I didn't read very carefully.) Yes, --format is a global parameter. >> >> * ovs-appctl adds this paramater as an argument to a 'params' JSON array. >> i.e. separate element in the array to every command it executes, if >> provided. >> >> * Server (ovs-vswitchd, ovsdb-server, etc.) receives the request, finds >> that argument, removes it from the list of parameters before passing >> to the actual handler. This would mean that ovs-appctl would parse the '--format' option and the server would have to parse it again, this time from the JSON array ('params'). This is what I tried to express above: The server would have to implement this option parsing in 'process_commands()' even though ovs-appctl already had parsed the user input. Besides duplicating work (parsing twice) you would have to duplicate code, because getopt* (which is used in appctl) cannot be used on the server side. It cannot be, because getopt* has global state and thus it is not safe in general. >> >> * Before calling a handler, server stores these special parameters >> into struct unixctl_conn (maybe a sub-struct inside it, but it doesn't >> really matter). struct unixctl_conn is only declared in unixctl.h, command callbacks do not have access to its definition. Which makes sense because it is an implementation detail which command callbacks do not have to care about. In contrast, adding a separate argument, whether it is an plain value or a wrapped in a (new) struct, highlights the intent (=> consumable by callback). >> >> * Handler always has the conn pointer. Handler can ask in which format >> the user prefers the output and generate one. E.g.: >> >> if (unixctl_command_reply_format(conn) == UNIXCTL_REPLY_FMT_JSON) { >> unixctl_command_reply_json(conn, ...); >> } else { >> unixctl_command_reply(conn, ...); >> } >> >> Notice that nothing bad happens if command is not aware of the formats, >> because unixctl_command_reply() is a JSON reply, but with a simple >> JSON string instead of any fancy structure. We talked about how to handle cases where a user asks for an unsupported format before. IIRC the consensus was that a command is supposed to fail if the user asks for JSON output but it has not been implemented (yet). Silently ignoring this and just returning text output instead is against common practice. >> >> * Result is sent back to the ovs-appctl. >> >> * If the --format=json was originally requested, just dump the result >> as JSON to the output in the unixctl_client_transact(). If format >> wasn't requested, parse it and print out the same way as it is done >> right now, if the result is not a simple JSON string - error. Not sure what you mean with your second sentence. What I do is dump whatever is returned. If it is a simple JSON string, print it as we did before. If it is JSON object or JSON array, it is also just printed (in future revision: pretty-printed if requested by user). If it is anything else then fail as we did before. >> >> This approach requires no changes in the code that doesn't want to provide >> JSON output and no changes to the command registering API. Because you are cheating: Your approach turns struct unixctl_conn from an implementation detail into a part of the public API 😋 Suddenly the developer of the callback(s) also have access to the other parameters in unixctl_conn. I would prefer to keep unixctl_conn as is and instead introduce a new argument which makes our intent clear. >> All commands support JSON output automatically. But commands that do >> not handle it specially will just return a simple JSON string. We will >> print it out as a JSON string, there is nothing bad in this. In that sense, all commands already support JSON output! The result of unixctl_command_reply is a JSON string already. >> >> Prettyfication can be done on the ovs-appctl side by an extra argument >> that doesn't need to be transferred to the server (already done this way). Agree, will implement it in a follow up. >> >> The only issue is that it's hard to tell what happened when a new >> ovs-appctl --format=json communicates with the old server and gets a >> number of arguments error. But we could add a hint 'Can be caused by an >> older <target> not supporting JSON format' to every such reply if format >> was requested and let user try without format and get a proper error if >> there is one. In any case, that should not be a frequent case, as I >> would expect the most users of JSON format will be scripts with their >> own JSON-RPC implementation, not ovs-appctl. >> >> Thoughts? In my current implementation, ovs-appctl is able to detect when a (old) server does not support the new API. It will fail with a proper error message, telling the user to upgrade its server implementation. If we keep the JSON-RPC API as it is today, there is no nice way of detecting old servers. However, I am still thinking about your approach. Keeping the JSON-RPC API as it is today, has the downsides listed above (messier code, no nice way of handling backward compat to older servers), but the over-the-socket API would be cleaner. Would be nice to hear some other people's opinion on the tradeoffs. >> Some more comments below. >> >>> For (2.) see below. >>> >>>> unixctl_command_register() in lib/unixctl.* has been cloned as >>>> unixctl_command_register_fmt() in order to demonstrate the new API. >>>> The latter will be replaced with the former in a later patch. The >>>> new function gained an int argument called 'output_fmts' with which >>>> commands have to announce their support for output formats. >>>> >>>> Function type unixctl_cb_func in lib/unixctl.h has gained a new arg >>>> 'enum ovs_output_fmt fmt'. For now, it has been added as a comment >>>> only for the huge changes reason mentioned earlier. The output format >>>> which is passed via unix socket to ovs-vswitchd will be converted into >>>> a value of type 'enum ovs_output_fmt' in lib/unixctl.c and then passed >>>> to said 'fmt' arg of the choosen command handler (in a future patch). >>>> When a requested output format is not supported by a command, >>>> then process_command() in lib/unixctl.c will return an error. >>> (2.) Instead of adding 'enum ovs_output_fmt fmt' as a new arg to function >>> type unixctl_cb_func, an indirection will be used: >>> >>> The enum value will be put into a struct and the struct will be added to >>> unixctl_cb_func. This would allow us to add new options easily without >>> having to change all command callbacks again. >> We already have strunc unixctl_conn where we can store generic things >> and have functions-accessors. Discussed above. >> >>> Wdyt? >>> >>>> This patch does not yet pass the choosen output format to commands. >>>> Doing so requires changes to all unixctl_command_register() calls and >>>> all command callbacks. To improve readibility those changes have been >>>> split out into a follow up patch. Respectively, whenever an output >>>> format other than 'text' is choosen for ovs-xxx tools, they will fail. >>>> By default, the output format is plain-text as before. >>>> >>>> In popular tools like kubectl the option for output control is usually >>>> called '-o|--output' instead of '-f,--format'. But ovs-appctl already >>>> has an short option '-o' which prints the available ovs-appctl options >>>> ('--option'). The now choosen name also better alines with ovsdb-client >>>> where '-f,--format' controls output formatting. >>>> >>>> Reported-at: https://bugzilla.redhat.com/1824861 >>>> Signed-off-by: Jakob Meng <[email protected]> >>>> --- >>>> lib/command-line.c | 36 ++++++ >>>> lib/command-line.h | 10 ++ >>>> lib/dpctl.h | 4 + >>>> lib/unixctl.c | 260 ++++++++++++++++++++++++++++++++--------- >>>> lib/unixctl.h | 17 ++- >>>> tests/pmd.at | 5 + >>>> utilities/ovs-appctl.c | 65 ++++++++--- >>>> utilities/ovs-dpctl.c | 12 ++ >> Don't touch interface of the ovs-dpctl binary. It doesn't need to have >> JSON output. ovs-appctl dpctl/ should pretty much always be used instead. >> >> Best regards, Ilya Maximets. What interface should not be touched? CLI args of ovs-dpctl? Or ovs-dpctl in general? Gruß, Jakob _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
