jrevels opened a new issue, #367:
URL: https://github.com/apache/arrow-julia/issues/367
It seems like `Union{ZonedDateTime,Missing}` (de)serialization can bork
Arrow.jl in some circumstances, presumably to a misapplied `Arrow.NullVector`
optimization (or code in that optimization assuming a constructor exists):
```jl
julia> using Arrow, TimeZones
julia> t = (; x=Union{ZonedDateTime,Missing}[missing])
(x = Union{Missing, ZonedDateTime}[missing],)
julia> a = Arrow.Table(Arrow.tobuffer(t))
Arrow.Table with 1 rows, 1 columns, and schema:
:x ZonedDateTime
julia> a.x[1]
ERROR: MethodError: no method matching ZonedDateTime(::Missing)
Closest candidates are:
ZonedDateTime(::Union{Dates.Period, TimeZone}...) at
~/.julia/packages/TimeZones/2oXvO/src/types/zoneddatetime.jl:136
ZonedDateTime(::AbstractString) at
~/.julia/packages/TimeZones/2oXvO/src/parse.jl:86
ZonedDateTime(::AbstractString, ::AbstractString; locale) at
~/.julia/packages/TimeZones/2oXvO/src/parse.jl:100
...
Stacktrace:
[1] fromarrow(#unused#::Type{ZonedDateTime}, x::Missing)
@ ArrowTypes ~/.julia/packages/ArrowTypes/dkiHE/src/ArrowTypes.jl:157
[2] getindex(v::Arrow.NullVector{ZonedDateTime}, i::Int64)
@ Arrow ~/.julia/packages/Arrow/QsQ3U/src/arraytypes/arraytypes.jl:99
[3] top-level scope
@ REPL[13]:1
```
Note that if we insert an actual `ZonedDateTime` and avoid the `NullVector`
optimization:
```jl
julia> t = (; x=Union{ZonedDateTime,Missing}[missing,
ZonedDateTime("0001-01-01T00:00:00.000Z")])
(x = Union{Missing, ZonedDateTime}[missing, ZonedDateTime(1, 1, 1, tz"Z")],)
julia> a = Arrow.Table(Arrow.tobuffer(t))
Arrow.Table with 2 rows, 1 columns, and schema:
:x Union{Missing, ZonedDateTime}
julia> a.x[1]
missing
julia> a.x[2]
0001-01-01T00:00:00+00:00
```
I tried this MWE with other types besides `ZonedDateTime` (`Int`, `String`,
`NamedTuple`) and none of those variants had this issue (and none of the
resulted in an `Arrow.NullVector`, but instead in an
`Arrow.Primitive`/`Arrow.List`/`Arrow.Struct`, all with appropriate
`Union{Missing,T}` eltype.
So I guess something special is going on with `ZonedDateTime`. I'll take a
peek.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
