Hi Mike,
Each layer in the stack ought to have its own test suite, comprehensive (as
best we can) set of test cases, and kicked off via CI (and on-demand from the
CLI via `sbt test`). In this case. The test suite will be communicating with
the debugger over the DAP protocol and asserting expectations. It doesn't
require a front-end or human interaction.
On 7/20/22, 10:18 AM, "Mike Beckerle" <mbecke...@apache.org> wrote:
I do have one further argument opposed to making it part of the CLI.
To me, if it cannot be tested in isolation from the VSCode front end, then
we should not make it part of Daffodil proper.
So are there tests and test-rig for exercising the dap server in isolation?
If not I think any PR to add it to the daffodil CLI should get -1. We have
a general principle that tests are required for things added to the
daffodil code base.
Furthermore, test coverage is supposed not to be falling, but increasing
over time.
Once there are tests for it that will confirm it operating properly in
isolation from the front end, then I would not have an objection to adding
it to daffodil's CLI.
I suspect we're going to need to enhance the dap server to have some sort
of capture capability so that manual testing of the front end can be
captured in terms of the commands sent to the dap server, and the responses
received from it, and those can be then turned into test cases to drive the
dap server in isolation and compare actual with expected responses.
...mikeb
On Wed, Jul 20, 2022 at 9:40 AM Steve Lawrence <slawre...@apache.org> wrote:
>
> I'm not sure I'm convinced by these arguments (note that I don't feel
> strongly either way, though I lean towards including in the repo).
> Comments inline.
>
> On 7/20/22 9:04 AM, Mike Beckerle wrote:
> > I think putting dap into the main daffodil repo is a false economy.
> > Consider:
> >
> > 1) more sbt hacks to exclude the tests of the dap from 'sbt test' and to
> > add an alternative 'sbt dap:test' which runs those tests.
>
> We could just put them as part of the IntegratoonTest if we don't want
> to run them as normal tests. IntegrationTests already take forever, and
> "sbt testOnly" can run a subset of tests if you only want to test DAP
> stuff. Note that we also don't even have any DAP tests right now so I'm
> not even sure what mechanisms they would use or how slow/fast they would
> be, so it might not actually add a significant amount of time to testing.
>
> > 2) that won't work with IDEs automatically, so they will be unable to
run
> > the dap tests without renaming directories or such.
> > (CLI has this same problem today. I do not yet have a convenient
> workaround
> > for this, and it is a pain in the neck routinely)
>
> I didn't know integration tests couldn't be run with IDE's. I think
> that's worth solving, and the solution should work the same for DAP tests.
>
> > 3) another module inside daffodil to compile when 'sbt compile' is
> issued.
> > So more delay in the routine edit-compile-debug cycle.
>
> From a clean/compile, it takes me on average about 4 seconds to compile
> the server. Compared to Daffodil, which takes me about 40 seconds. It's
> not nothing, but I'm not sure I would notice a 4 second difference. And
> it wouldn't surprise me if that's mostly Scala/Sbt overhead anyways.
> There's only 2000 lines of DAP code, that should compile pretty fast.
>
> > 4) more IDE customizations to save that extra compilation time when
doing
> > IDE builds.
>
> I'm not sure I understand what this means, but there shouldn't be any
> extra IDE configuration, it should work the same way any other
> subproject works.
>
> >
> > The above is a huge hassle to maintain compared with one more repo that
> is
> > completely ordinary with respect to build configuration, IDE support,
> etc.
>
> I would argue we could make it completely ordinary. It's just another
> Daffodil subproject.
>
> > Creating a separate repo has none of these drawbacks. 'sbt test' can
work
> > normally to test it. Tests live in src/test as normal. No need for
> special
> > exclusions or sbt hackery.
> > IDEs will do the right thing by default.
>
> Another tought, from a testing perspective I think we're much less
> likely to accidentally break DAP if it's integrated in Daffodil. I know
> I'm very rarely going to want to run "sbt publishLocal" Daffodil, modify
> daffdoil-debugger to use a SNAPSHOT version, and then run "sbt test"
> from there. We have this same issue with DFDL schemas where we rearely
> test schemas because they're in a separate repo and only find out when
> we broke something because of some nightly test or someone starts using
> the schema.
>
> Maybe some GitHub action trigger could do this for Daffodil changes, or
> someone include a publishLocal, download debugger and test in the GitHub
> Actions, but that's just more complication.
>
>
> This also potentially adds extra complication to the release process.
> I'm not even sure how it would work. Would we first do a Daffodil
> release (discuss, vote, publish), and then update the daffodil-debugger
> repo to point to the new daffodil version and then do another release
> for that (discuss, vote, publish)? So essentially two separate release
> processes? Or do we always do them at the same time and the
> release-candidate script builds both Daffodil and the debugger, and the
> vote/release is for both? Whatever the case, this probably requires
> changes/maintenance to the release-candidate tool, and is a change to
> our current process. I'm sure we can manage, but if we integrate it into
> the main repo, nothing with the release process changes.
>
-----------------------------------------------------------------
This message and any files transmitted within are intended
solely for the addressee or its representative and may contain
company proprietary information. If you are not the intended
recipient, notify the sender immediately and delete this
message. Publication, reproduction, forwarding, or content
disclosure is prohibited without the consent of the original
sender and may be unlawful.
Concurrent Technologies Corporation and its Affiliates.
www.ctc.com 1-800-282-4392
-----------------------------------------------------------------
