Regardless of the amount of work involved with writing a test
harness/driver (which could be mostly in Python as one possibility), I
find it hard to believe that writing functional tests that look like
this->CheckUnary("utf8_reverse", "[\"ɑɽⱤæÆ&\", null, \"\", \"bbb\"]",
this->type(),
"[\"&ÆæⱤɽɑ\", null, \"\", \"bbb\"]");
is the best that we can do. There are various issues:
* Adding more tests requires recompilation each time, and the larger
the test suite, the longer our compile times are. Interpreted tests
have no compile-time cost.
* Specifying data inputs as string literals in C++ is more complex
than in JSON or Python, say.
* Tests require interacting with various C++ specific constructs like
data types, and utilizing parametric test structures in Googletest to
run the same tests across data types (like utf8/large_utf8) that we
expect to behave similarly.
* To Julian's point, the tests are C++ specific
There's other C++-specific requirements, like:
* It must not be too difficult to use gdb to set breakpoints when debugging
In an ideal world the interpreted test harness for C++ would not
require building pyarrow (since this would make things more difficult
for people who are just interacting with the C++ project).
On Fri, May 14, 2021 at 12:56 PM Weston Pace <[email protected]> wrote:
>
> With that in mind it seems the somewhat recurring discussion on coming
> up with a language independent standard for logical query plans
> (https://lists.apache.org/thread.html/rfab15e09c97a8fb961d6c5db8b2093824c58d11a51981a40f40cc2c0%40%3Cdev.arrow.apache.org%3E)
> would be relevant. Each test case would then be a triplet of (Input
> Dataframe, Logical Plan, Output Dataframe). So perhaps tackling this
> effort would be to make progress on both fronts.
>
> On Fri, May 14, 2021 at 7:39 AM Julian Hyde <[email protected]> wrote:
> >
> > Do these any of these compute functions have analogs in other
> > implementations of Arrow (e.g. Rust)?
> >
> > I believe that as much as possible of Arrow’s compute functionality should
> > be cross-language. Perhaps there are language-specific differences in how
> > functions are invoked, but the basic functionality is the same.
> >
> > If people buy into that premise, then a single suite of tests is a powerful
> > way to make that happen. The tests can be written in a high-level language
> > and can generate tests in each implementation language. (For these
> > purposes, the “high-level language” could be a special text format, could
> > be a data language such as JSON, or could be a programming language such as
> > Python; it doesn’t matter much.)
> >
> > For example,
> >
> > assertThatCall(“foo(1, 2)”, returns(“3”))
> >
> > might actually call foo with arguments 1 and 2, or it might generate a C++
> > or Rust test that does the same.
> >
> >
> > Julian
> >
> >
> > > On May 14, 2021, at 8:45 AM, Antoine Pitrou <[email protected]> wrote:
> > >
> > >
> > > Le 14/05/2021 à 15:30, Wes McKinney a écrit :
> > >> hi folks,
> > >> As we build more functions (kernels) in the project, I note that the
> > >> amount of hand-coded C++ code relating to testing function correctness
> > >> is growing significantly. Many of these tests are quite simple and
> > >> could be expressed in a text format that can be parsed and evaluated.
> > >> Thoughts about building something like that to make it easier to write
> > >> functional correctness tests?
> > >
> > > Or perhaps build-up higher level C++ functions if desired?
> > >
> > > Or even write some of those tests as part of the PyArrow test suite.
> > >
> > > I'm not sure adding a custom (and probably inflexible) text format is
> > > really a good use of our time.
> > >
> > > Regards
> > >
> > > Antoine.
> >
