On Tue, 28 May 2024 at 14:37, Andrew Lamb <[email protected]> wrote:
> One thing we could that might move the burden on to implementations rather > than some central CI job (which is a substantial effort, I agree, having > worked with the arrow ne) > > Have you any slides/docs on the experience here? > Perhaps we could start simply with "reader compatibility" with the > existing files in parquet-testing[1] > > 1. Define a JSON file format with expected results > 2. Document how readers should generate that expected JSON file > Then to determine compatibility with each "feature" an implementation would > show it could read and create the expected JSON file. > > This misses plenty of potential nuance, but it would likely cover most of > the basic "can this implementation read files" type questions > forcing generation of different encodings might be tricky, at least if the library is making that decision for you. Doing something like this for the existing test files would work as a PoC and decouple test framework design and wire up from everything else > >
