patcollis34 opened a new issue #12554:
URL: https://github.com/apache/arrow/issues/12554
Hey all,
90% sure this isn't the place for this question, but couldn't find a gitter
or slack to post for this project, possibly JIRA but I've never used that
before.
Wondering if the plasma datastore would/should be the recommended way to
serve arrow(numpy arrays) in a distributed read-only environment(1 pod write,
many pod read). In my first hack code I ran into ObjectID overwrite problems as
well as the 20 bytes key limitation, which led me to think plasma wasn't fully
suited for this workload.
Current setup: syncing local LMDBs per pod with a master efs, but has a few
problems(serialiazation times, unnecessary syncs, etc..).
Any inputs or suggestions here is much appreciated
Also happy to delete this comment and move it to the proper forum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
