paleolimbot commented on issue #478:
URL:
https://github.com/apache/arrow-nanoarrow/issues/478#issuecomment-2119492920
I'm glad this is useful!
We're about to release the current bindings and unfortunately I don't think
we can get out-of-the-box support for this before the release (there will
probably be another release in early July).
Getting the details right for the full matrix of Arrow date/time/datetime
vs. Python date/time/datetime objects s hard; however, if you have full control
over the datetime objects you are producing, the workaround is fairly compact
(below).
```python
import datetime
import nanoarrow as na
def gen_name():
for i in range(10):
yield "John Doe"
def gen_age():
for i in range(10):
yield 34
def gen_timestamp():
for i in range(10):
yield datetime.datetime.now().timestamp()
def gen_timestamp_int_us(timestamps):
for item in timestamps:
yield None if item is None else int(item * 1000)
def to_arrow():
# Declare schema with the actual arrow type (timestamp)
schema = na.struct(
{"name": na.string(), "age": na.int64(), "timestamp":
na.timestamp("ms")}
)
# Create column as an int64 array with storage values
columns = [
na.c_array(gen_name(), na.string()),
na.c_array(gen_age(), na.int64()),
na.c_array((int(t * 1e3) for t in gen_timestamp()), na.int64()),
]
# Skip validation when creating from buffers
return na.c_array_from_buffers(
schema,
length=columns[0].length,
buffers=[None],
children=columns,
validation_level="none",
)
na.Array(to_arrow())
```
If you'd like to help add support, one way would be to add a method to the
`ArrayFromIterableBuilder`:
https://github.com/apache/arrow-nanoarrow/blob/c413d69f78eedefac378a318390e808b5a16e6b9/python/src/nanoarrow/c_array.py#L481-L484
then add a line in the mapping that you linked with a mapping from
`CArrowType.TIMESTAMP` to the name of the method you added:
https://github.com/apache/arrow-nanoarrow/blob/c413d69f78eedefac378a318390e808b5a16e6b9/python/src/nanoarrow/c_array.py#L531-L535
Getting the details right with repsect to timezones and units is hard, but
is essentially a reverse-engineered version of the conversion in the other
direction:
https://github.com/apache/arrow-nanoarrow/blob/c413d69f78eedefac378a318390e808b5a16e6b9/python/src/nanoarrow/iterator.py#L382-L403
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
