svilupp opened a new issue, #335: URL: https://github.com/apache/arrow-julia/issues/335
First of all, thank you for the amazing package! I have noticed unexpected behaviour that I wanted to point out. **Expected behaviour:** rational numbers like 1.0 and 0.1 will be represented as Float; they can be saved and loaded again. **Actual behaviour:** When writing column with eltype Decimals.Decimal, `Arrow.write(filename,df)` will give a method error (see below) and `Arrow.write(filename,df;compress=:lz4)` will complete without an error, but the resulting table is wrong when re-read (see MWE below). I've had a quick look at the code base and I cannot see any type checks - are those left to the user / MethodErrors? MWE: ``` using Decimals using DataFrames, Arrow df=DataFrame(:a=>[Decimal(2.0)]) # this will fail with error that Decimal cannot be saved Arrow.write("test.feather", df) # nested task error: MethodError: no method matching write(::IOBuffer, ::Decimals.Decimal) # this will succeed Arrow.write("test.feather", df;compress=:lz4) # but the loaded dataframe will be rubbish df2=Arrow.Table("test.feather")|>DataFrame # 1×1 DataFrame # Row │ a # │ Float64 # ─────┼───────────── # 1 │ 2.1509e-314 ``` Error stack trace from Arrow.write() without a keyword argument: > ERROR: TaskFailedException Stacktrace: [1] wait @ ./task.jl:345 [inlined] [2] close(writer::Arrow.Writer{IOStream}) @ Arrow ~/.julia/packages/Arrow/ZlMFU/src/write.jl:230 [3] open(::Arrow.var"#120#121"{DataFrame}, ::Type, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:file,), Tuple{Bool}}}) @ Base ./io.jl:386 [4] #write#119 @ ~/.julia/packages/Arrow/ZlMFU/src/write.jl:57 [inlined] [5] write(file_path::String, tbl::DataFrame) @ Arrow ~/.julia/packages/Arrow/ZlMFU/src/write.jl:56 [6] top-level scope @ REPL[14]:1 > nested task error: MethodError: no method matching write(::IOBuffer, ::Decimals.Decimal) Closest candidates are: write(::IO, ::Any) at io.jl:672 write(::IO, ::Any, ::Any...) at io.jl:673 write(::Base.GenericIOBuffer, ::UInt8) at iobuffer.jl:442 ... Stacktrace: [1] write(io::IOBuffer, x::Decimals.Decimal) @ Base ./io.jl:672 [2] writearray(io::IOStream, #unused#::Type{Decimals.Decimal}, col::Vector{Union{Missing, Decimals.Decimal}}) @ Arrow ~/.julia/packages/Arrow/ZlMFU/src/utils.jl:50 [3] writebuffer(io::IOStream, col::Arrow.Primitive{Union{Missing, Decimals.Decimal}, Vector{Union{Missing, Decimals.Decimal}}}, alignment::Int64) @ Arrow ~/.julia/packages/Arrow/ZlMFU/src/arraytypes/primitive.jl:102 [4] write(io::IOStream, msg::Arrow.Message, blocks::Tuple{Vector{Arrow.Block}, Vector{Arrow.Block}}, sch::Base.RefValue{Tables.Schema}, alignment::Int64) @ Arrow ~/.julia/packages/Arrow/ZlMFU/src/write.jl:365 [5] macro expansion @ ~/.julia/packages/Arrow/ZlMFU/src/write.jl:149 [inlined] [6] (::Arrow.var"#122#124"{IOStream, Int64, Tuple{Vector{Arrow.Block}, Vector{Arrow.Block}}, Base.RefValue{Tables.Schema}, Arrow.OrderedChannel{Arrow.Message}})() @ Arrow ./threadingconstructs.jl:258 **Package version** [69666777] Arrow v2.3.0 [a93c6f00] DataFrames v1.3.4 [194296ae] LibPQ v1.14.0 **versioninfo()** (but it was the same on 1.7) Julia Version 1.8.0 Commit 5544a0fab76 (2022-08-17 13:38 UTC) Platform Info: OS: macOS (arm64-apple-darwin21.3.0) CPU: 8 × Apple M1 Pro WORD_SIZE: 64 LIBM: libopenlibm LLVM: libLLVM-13.0.1 (ORCJIT, apple-m1) Threads: 6 on 6 virtual cores -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org