Thank you for drafting!
I gave a few comments in the community call last Wednesday, but I'll
echo them here for discussion. None of these are blocking concerns,
just thoughts having seen this particular concept be reinvented in a
lot of places.
While this does a great job of losslessly translating flatbuffer
schemas into JSON and back again, flatbuffer schemas are rather
awkwardly laid out and the human typability/readability is not great.
I think there is an opportunity for this JSON spec to replace a lot of
bespoke data type / field / schema representations, but I worry that
it's a little verbose in its current form to fill that niche.
Just as an example, a schema with a single uint32 column with this proposal is:
{"arrowSchema": "1.5.0", "fields": [{"name": "col0", "nullable": true,
"type": {"name": "int", "bitWidth": 32, "isSigned": false}}]}
...whereas with a bit of language (apply defaults for nullability,
optional/independent versioning, special case unparameterized types)
this could be more like:
[{"name": "col0", "type": "uint32"}]
...drawing from BigQuery [1] here), although that does open up a
significantly more surface area for bikeshedding on which strings map
to which type (unless we want to reuse the C data interface ones,
which from the proposal it seems like maybe we don't).
Another opportunity to reduce verbosity is metadata, which for an
extension type with JSON metadata would be
[{"key": "ARROW:extension:name", "value": "geoarrow.wkb"}, {"key":
"ARROW:extension:metadata", "value": "{\"crs\": \"OGC:CRS84\"}"}]
...and could be more readable:
{"ARROW:extension:name": "geoarrow.wkb", "ARROW:extension:metadata":
{"crs": "OGC:CRS84"}}
Basically, if the goal is human readability, I think it's worth the
effort to do the bikeshedding on how to represent these concepts that
mirror how they are printed/enumerated in implementations (not
necessarily the .fbs files). If the goal is machine readability within
existing arrow implementations, perfectly mirroring flatbuffers JSON
or the C data interface would be less work (more opportunity for
reusing existing functions).
Cheers,
-dewey
[1]
https://docs.cloud.google.com/ruby/docs/reference/google-cloud-bigquery/latest/Google-Cloud-Bigquery-Schema#Google__Cloud__Bigquery__Schema_load_class_
On Thu, Jul 2, 2026 at 3:37 PM Kent Wu <[email protected]> wrote:
>
> Hi all,
>
> I'd like to raise a topic for discussion that has surfaced a few
> separate times over the years in the Arrow community, which is the
> lack of a canonical human-readable representation of Arrow schemas.
>
> Arrow schemas today only canonically serialize as IPC binary, which is
> a friction point for application-level tasks where binary blobs are
> not ergonomic, such as JSON API contracts, hand authoring or reading,
> and persistence use cases, to name a few.
>
> The motivating pain point right now is at the ADBC metadata boundary.
> The 1.2 milestone includes proposals for new metadata APIs, and
> several contributors have noted that returning Arrow schemas via IPC
> is unsatisfactory for this use case.
>
> ADBC 1.2 Milestone
> - https://github.com/apache/arrow-adbc/milestone/9
>
> Discussion within various issues:
> - https://github.com/apache/arrow-adbc/issues/4400
> - https://github.com/apache/arrow-adbc/issues/1514
> - https://github.com/apache/arrow-adbc/issues/1704
> - https://github.com/apache/arrow-adbc/pull/4031
>
> As a starting point, I've put together a proposal for how a JSON
> representation might be structured.
>
> Rather than dump the whole thing here, I've put it in this google doc
> which is open for comments:
>
> https://docs.google.com/document/d/1ho0FKy9ge0tUSRzebq1AFi28KR1H_utecQjHx4CNCEs/edit?usp=sharing
>
> It's early and open to change, so I'd welcome feedback of any kind.
>
> Looking forward to hearing your thoughts
