[ https://issues.apache.org/jira/browse/ARROW-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438555#comment-16438555 ]
Kouhei Sutou commented on ARROW-2457: ------------------------------------- Do you have a program that reproduces this case? > garrow_array_builder_append_values() won't work for large arrays > ---------------------------------------------------------------- > > Key: ARROW-2457 > URL: https://issues.apache.org/jira/browse/ARROW-2457 > Project: Apache Arrow > Issue Type: Bug > Components: C, C++, GLib > Affects Versions: 0.8.0, 0.9.0 > Reporter: Haralampos Gavriilidis > Assignee: Kouhei Sutou > Priority: Major > > I am using garrow_array_builder_append_values() to transform a native C array > to an Arrow array, without calling arrow_array_builder_append multiple times. > When calling garrow_array_builder_append_values() in array-builder.cpp with > following signature: > {code:java} > garrow_array_builder_append_values(GArrowArrayBuilder *builder, > const VALUE *values, > gint64 values_length, > const gboolean *is_valids, > gint64 is_valids_length, > GError **error, > const gchar *context) > {code} > it will fail for large arrays. This is probably happening because the > is_valids array is copied to the valid_bytes array (of different type), for > which the memory is allocated on the stack, and not on the heap, like shown > on the snippet below: > {code:java} > uint8_t valid_bytes[is_valids_length]; > for (gint64 i = 0; i < is_valids_length; ++i){ > valid_bytes[i] = is_valids[i]; > } > {code} > A way to avoid this problem would be to allocate memory for the valid_bytes > array using malloc() or something similar. Is this behavior intended, maybe > because no large arrays should be handed over to that function, or it is > rather a bug? -- This message was sent by Atlassian JIRA (v7.6.3#76005)