Arrow uses flatbuffers under the hood. https://google.github.io/flatbuffers/
FlatBuffers is an efficient cross platform serialization library for C++, C#, C, Go, Java, Kotlin, JavaScript, Lobster, Lua, TypeScript, PHP, Python, Rust and Swift. It was originally created at Google for game development and other performance-critical applications. -----Original Message----- From: Robert Bigelow <rfbige...@gmail.com> Sent: Sunday, December 13, 2020 1:00 PM To: dev@arrow.apache.org Subject: arrow for game engine / graphics workloads? External Email: Use caution with links and attachments Dear Arrow devs, I’m writing a game engine in Swift, and the next system to design is the resource manager / asset database. Arrow seems like an attractive option for the heart of an engine, since many of the performance goals stated for analytic workloads are shared by real-time rendering. Data layout is extremely important, and I’d like to be able to feed the renderer without chasing pointers. So far my plan is to create a custom format for the asset database that will be used at runtime. I plan on having a tool that traverses a scene graph write this format, then have the resource manager use mmap to load assets. Arrow seems like a good fit for such a format, as (if I understand correctly) only the Schema needs to be deserialized before the data would be available, and it could be used to back streaming APIs. Do you know of any work being done with Arrow in the real-time rendering or game engine space? Would the API presented by Arrow be a good fit, assuming I’d mostly need to expose buffers of typed data to feed the renderer? One final question assuming none of your answers have dissuaded me. Would the C/glib Arrow library be reasonable, since Swift can import C headers? It seems that no intrepid Swift developers have started a native Swift implementation yet. Thanks for your time, and kudos for your work on the Arrow project. It is very impressive! Cheers, Rob Bigelow This message may contain information that is confidential or privileged. If you are not the intended recipient, please advise the sender immediately and delete this message. See http://www.blackrock.com/corporate/compliance/email-disclaimers for further information. Please refer to http://www.blackrock.com/corporate/compliance/privacy-policy for more information about BlackRock’s Privacy Policy. For a list of BlackRock's office addresses worldwide, see http://www.blackrock.com/corporate/about-us/contacts-locations. © 2020 BlackRock, Inc. All rights reserved.
