In my opinion, the nature of DFDL makes it much more amenable to a read-only time-travel style of debugging. We kind of already support this with the --trace functionality, but the UI there (a giant text dump) is less then ideal. The mininimal viable product for this would be a reader for such a trace dump). Everything beyond that point is enhancements. With all the enhancements we fully blur the line between glorified log viewer and full time-traveling debugger.
In addition to the debug environment being based around time travel; the Daffodil runtime itself is based around time travel (backtracing for parse, and suspensions for unsparse). There are additional mismatches between an ideal DFDL debugging environment and a normal debugger. With most programming langauges the central objects for debugging are variables and the stacktrace. For DFDL, the central objects are the infoset and datastream. The main benefit I see with an IDE integration are easier integrations with a source code viewer/editor. This is no small thing, but is also our lowest priority and delegated to phase 4. I also see this as an excersise in learning how a DFDL debug environment ought to look like. Since we know (or, at least I think) a traditional debug view does not match well, I would prefer that we avoid fighting with an IDEs notion before we know what it is we want. What we might want to re-evaluate our plans at phase 3. This is when we will be adding the communication protocal an interactive debugger would need to use. At this point we can ask if it is easier to get the IDE to understand how we want to display information, or if it is easier to implement/integrate whatever IDE functionality we want into our own program. Hopefully at that point, we have a better idea of what it is we are building. ________________________________ From: John Wass <jwa...@gmail.com> Sent: Tuesday, February 16, 2021 7:31 AM To: Sloane, Brandon <bslo...@owlcyberdefense.com> Cc: dev@daffodil.apache.org <dev@daffodil.apache.org> Subject: Re: Possible road map for GUI debugger > I am conceiving of it as a standalone utility. I'm interested in the reasoning there. No strong opinions, I can see some advantages either way, but an IDE integration does seem to have a head start in several areas related to debugging. On Mon, Feb 15, 2021 at 1:27 PM Sloane, Brandon <bslo...@owlcyberdefense.com<mailto:bslo...@owlcyberdefense.com>> wrote: We are at the very early stages. I am conceiving of it as a standalone utility. Brandon Sloane | Engineer [cid:177aacd632cba97c0f91] bslo...@owlcyberdefense.com<mailto:bslo...@owlcyberdefense.com> Connect with us! [cid:177aacd632cd46063f02]<https://www.linkedin.com/company/owlcyberdefense/>[cid:177aacd632c9d1c8fa93]<https://twitter.com/owlcyberdefense> [cid:177aacd632cc519938a4]<https://owlcyberdefense.com/resources/events/> The information contained in this transmission is for the personal and confidential use of the individual or entity to which it is addressed. If the reader is not the intended recipient, you are hereby notified that any review, dissemination, or copying of this communication is strictly prohibited. If you have received this transmission in error, please notify the sender immediately ________________________________ From: John Wass <jwa...@gmail.com<mailto:jwa...@gmail.com>> Sent: Monday, February 15, 2021 7:39 AM To: dev@daffodil.apache.org<mailto:dev@daffodil.apache.org> <dev@daffodil.apache.org<mailto:dev@daffodil.apache.org>> Subject: Re: Possible road map for GUI debugger Is the gui envisioned to be an ide plugin or a standalone application (ie swing, fx, ...)? On Sun, Jan 24, 2021 at 2:08 PM Sloane, Brandon <bslo...@owlcyberdefense.com<mailto:bslo...@owlcyberdefense.com>> wrote: > Following up on the recent discussion about the Daffodil debugger, I would > like to propose a roadmap for creating graphical debugger. > Organizationally, the GUI debugger is a separate application (maintained > in the normal Daffodil repository) that may include any of the current > Daffodil libraries as a dependency. > > Phases 1 and 2 focus on developing an enhanced trace functionality; where > Daffodil outputs a machine readable trace of the parse/unparse process. The > GUI then allows a user to view this trace offline. > > Phase 3 adds support for interactive debugging, by allowing the > Daffodil-cli and Daffodil-gui-debugger to act in a client/server > relationship > > Phase 4 focuses on backend debugger enhancements. > > In more detail: > > Phase 1: > > - Add a --machine-trace flag to Daffodil CLI, that is analogous to the > current --trace option. Output is a stream of XML parse-state nodes, each > one describing the current state of the parse, including: > - Current infoset > - Current bit position > - Current parser > - A UID of the current state > - A UID of the parent state (in event of backtracking, multiple > states will have the same parent). > - Machine-trace output stream also include input-data nodes, that > include: > - A region of (byte aligned) input data (hex encoded?) > - The byte position of the start of the region > - The relative location of the input data nodes is undefined, but > given the complete trace, it should be possible to reconstruct the > original > dataset by stitching together the data in the nodes. The goal here is > that > we can just output data as Daffodil reads it in. > - --machine-trace take an optional argument of a file path. If > provided, its output will be directed to a file at said location (otherwise > stderr) > - Create a GUI program that can ingest the machine trace and display > it in a 4-pane view: > - Pane 1 - trace navigator > - Represents the entire parse in a tree format. Each parse state > corresponds to a node on the tree > - The user can select a node on the tree to load it into the > other panes > - Pane 2 - Infoset viewer > - Shows the complete infoset corresponding to the node selected > in pane 1. > - Indicates the "current" position of the parser within the > infoset > - Highlights changes in the infoset relative to the parent node. > - Pane 3 - binary viewer > - Shows a complete hex-dump of the input data, indicating > current location within the parse > - Pane 3 -- parser view > - Shows the current parser. Probably will be a dumping ground > for additional information that does not warrant the addition of a > new pane. > > > Phase 2: > > - Support showing suspensions on unparse > - Display additional data about the parse/unparse state: > - Delimeter stack > - Variables > - Support additional formats for displaying input data > - Text (of selectable encoding) > - Binary (not hex; actual 1s and 0s) > - Support generating traces through the API as well. > > Phase 2.5: > > Gather user feedback and iterate on the trace viewer functionality > > Phase 3: > > - Define protocol for the debugger to talk with the CLI. > - Debugger can load schema, and input data. > - Debugger can step through the parse/unparse > - Viewer built in phases 1 and 2 continually updates as the parse > proceeds > - User can send CLI debugger commands and get the textual response; > but only for the current parse/unparse state. > - User can set breakpoints (possibly using textual commands). Debugger > supports continue until breakpoint > > Phase 3.5: > > > - Not debugger related, but as long as the Daffodil CLI can operate as > a server, support server mode operations for non-debug use cases. > > > Phase 4: > > - Support time travel debugging, with an interface in both the CLI and > GUI debuggers > - Support associating a trace with schema source, and allowing a "go > to source" functionality where the user can navigate directly to the > location in source corresponding to the current parser > - Support dynamically editing/reloading schema and input data > > Thoughts? > Brandon Sloane | Engineer > > bslo...@owlcyberdefense.com<mailto:bslo...@owlcyberdefense.com> > > Connect with us! > > <https://www.linkedin.com/company/owlcyberdefense/> > <https://twitter.com/owlcyberdefense> > > <https://owlcyberdefense.com/resources/events/> > > > > The information contained in this transmission is for the personal and > confidential use of the individual or entity to which it is addressed. If > the reader is not the intended recipient, you are hereby notified that any > review, dissemination, or copying of this communication is strictly > prohibited. If you have received this transmission in error, please notify > the sender immediately > >
