I think there's value, though, in having a representation that is (1)
unambiguous and consistent (2) reasonably human-friendly and machine-friendly,
even if it's not the most compact. This would be useful for (REST) APIs and
ADBC, and I don't think the compactness of the representation is as important
there. What kinds of use cases are you envisioning?
FWIW, we debated the JSON metadata question - I kind of wanted this too, but it
raises too many questions about what happens to the canonical
representation/equality/serialization/round-tripping. (For example: we need to
stringify the JSON to pass it to the type parser, and a round-trip would
possibly present semantically equivalent JSON but different JSON strings. JSON
doesn't _disallow_ duplicate keys, so what if we get different behavior? etc.)
Ultimately the motivation (for me) is to use this in ADBC where we want to
represent Arrow schemas inside Arrow data (where serialized IPC schemas seem to
be unpalatable), and so being well-defined is most important (I don't really
want to handroll something that is only used in ADBC).
-David
On Wed, Jul 8, 2026, at 04:36, Dewey Dunnington wrote:
> Thank you for drafting!
>
> I gave a few comments in the community call last Wednesday, but I'll
> echo them here for discussion. None of these are blocking concerns,
> just thoughts having seen this particular concept be reinvented in a
> lot of places.
>
> While this does a great job of losslessly translating flatbuffer
> schemas into JSON and back again, flatbuffer schemas are rather
> awkwardly laid out and the human typability/readability is not great.
> I think there is an opportunity for this JSON spec to replace a lot of
> bespoke data type / field / schema representations, but I worry that
> it's a little verbose in its current form to fill that niche.
>
> Just as an example, a schema with a single uint32 column with this proposal
> is:
>
> {"arrowSchema": "1.5.0", "fields": [{"name": "col0", "nullable": true,
> "type": {"name": "int", "bitWidth": 32, "isSigned": false}}]}
>
> ...whereas with a bit of language (apply defaults for nullability,
> optional/independent versioning, special case unparameterized types)
> this could be more like:
>
> [{"name": "col0", "type": "uint32"}]
>
> ...drawing from BigQuery [1] here), although that does open up a
> significantly more surface area for bikeshedding on which strings map
> to which type (unless we want to reuse the C data interface ones,
> which from the proposal it seems like maybe we don't).
>
> Another opportunity to reduce verbosity is metadata, which for an
> extension type with JSON metadata would be
>
> [{"key": "ARROW:extension:name", "value": "geoarrow.wkb"}, {"key":
> "ARROW:extension:metadata", "value": "{\"crs\": \"OGC:CRS84\"}"}]
>
> ...and could be more readable:
>
> {"ARROW:extension:name": "geoarrow.wkb", "ARROW:extension:metadata":
> {"crs": "OGC:CRS84"}}
>
> Basically, if the goal is human readability, I think it's worth the
> effort to do the bikeshedding on how to represent these concepts that
> mirror how they are printed/enumerated in implementations (not
> necessarily the .fbs files). If the goal is machine readability within
> existing arrow implementations, perfectly mirroring flatbuffers JSON
> or the C data interface would be less work (more opportunity for
> reusing existing functions).
>
> Cheers,
>
> -dewey
>
> [1]
> https://docs.cloud.google.com/ruby/docs/reference/google-cloud-bigquery/latest/Google-Cloud-Bigquery-Schema#Google__Cloud__Bigquery__Schema_load_class_
>
> On Thu, Jul 2, 2026 at 3:37 PM Kent Wu <[email protected]> wrote:
>>
>> Hi all,
>>
>> I'd like to raise a topic for discussion that has surfaced a few
>> separate times over the years in the Arrow community, which is the
>> lack of a canonical human-readable representation of Arrow schemas.
>>
>> Arrow schemas today only canonically serialize as IPC binary, which is
>> a friction point for application-level tasks where binary blobs are
>> not ergonomic, such as JSON API contracts, hand authoring or reading,
>> and persistence use cases, to name a few.
>>
>> The motivating pain point right now is at the ADBC metadata boundary.
>> The 1.2 milestone includes proposals for new metadata APIs, and
>> several contributors have noted that returning Arrow schemas via IPC
>> is unsatisfactory for this use case.
>>
>> ADBC 1.2 Milestone
>> - https://github.com/apache/arrow-adbc/milestone/9
>>
>> Discussion within various issues:
>> - https://github.com/apache/arrow-adbc/issues/4400
>> - https://github.com/apache/arrow-adbc/issues/1514
>> - https://github.com/apache/arrow-adbc/issues/1704
>> - https://github.com/apache/arrow-adbc/pull/4031
>>
>> As a starting point, I've put together a proposal for how a JSON
>> representation might be structured.
>>
>> Rather than dump the whole thing here, I've put it in this google doc
>> which is open for comments:
>>
>> https://docs.google.com/document/d/1ho0FKy9ge0tUSRzebq1AFi28KR1H_utecQjHx4CNCEs/edit?usp=sharing
>>
>> It's early and open to change, so I'd welcome feedback of any kind.
>>
>> Looking forward to hearing your thoughts
