This is an excellent point. I could use Flatbuffers directly to define any custom format needed by the engine. The engine itself would need to use the same principles the Arrow devs have, which I guess is true of any data-intensive system. Thanks for your response!
> On Dec 14, 2020, at 11:24 AM, Lee, David <david....@blackrock.com.INVALID> > wrote: > > Arrow uses flatbuffers under the hood. > > https://google.github.io/flatbuffers/ > > FlatBuffers is an efficient cross platform serialization library for C++, C#, > C, Go, Java, Kotlin, JavaScript, Lobster, Lua, TypeScript, PHP, Python, Rust > and Swift. It was originally created at Google for game development and other > performance-critical applications. > > > -----Original Message----- > From: Robert Bigelow <rfbige...@gmail.com> > Sent: Sunday, December 13, 2020 1:00 PM > To: dev@arrow.apache.org > Subject: arrow for game engine / graphics workloads? > > External Email: Use caution with links and attachments > > > Dear Arrow devs, > > I’m writing a game engine in Swift, and the next system to design is the > resource manager / asset database. Arrow seems like an attractive option for > the heart of an engine, since many of the performance goals stated for > analytic workloads are shared by real-time rendering. Data layout is > extremely important, and I’d like to be able to feed the renderer without > chasing pointers. So far my plan is to create a custom format for the asset > database that will be used at runtime. I plan on having a tool that traverses > a scene graph write this format, then have the resource manager use mmap to > load assets. Arrow seems like a good fit for such a format, as (if I > understand correctly) only the Schema needs to be deserialized before the > data would be available, and it could be used to back streaming APIs. > > Do you know of any work being done with Arrow in the real-time rendering or > game engine space? Would the API presented by Arrow be a good fit, assuming > I’d mostly need to expose buffers of typed data to feed the renderer? > > One final question assuming none of your answers have dissuaded me. Would the > C/glib Arrow library be reasonable, since Swift can import C headers? It > seems that no intrepid Swift developers have started a native Swift > implementation yet. > > Thanks for your time, and kudos for your work on the Arrow project. It is > very impressive! > > Cheers, > Rob Bigelow > > > This message may contain information that is confidential or privileged. If > you are not the intended recipient, please advise the sender immediately and > delete this message. See > http://www.blackrock.com/corporate/compliance/email-disclaimers for further > information. Please refer to > http://www.blackrock.com/corporate/compliance/privacy-policy for more > information about BlackRock’s Privacy Policy. > > > For a list of BlackRock's office addresses worldwide, see > http://www.blackrock.com/corporate/about-us/contacts-locations. > > © 2020 BlackRock, Inc. All rights reserved.
