Thanks, Joe. Looking forward to seeing this come together. Neal
On Mon, Jul 3, 2023 at 11:29 AM Joe Marshall <joe.marsh...@nottingham.ac.uk.invalid> wrote: > Hi, > > I'm a pyodide developer amongst other things (webassembly cpython > intepreter) and I've got some PRs in progress on arrow relating to > webassembly support. I wondered if it might be worth discussing my broader > ideas for this on the list or at the biweekly development meeting? > > So far I have 35176 in, which makes arrow run on a single thread. This is > needed because in a lot of webassembly environments (browsers at least, > pyodide), threading isn't available or is heavily constrained. > > With that I've aimed to make it relatively transparent to users, so that > things like datasets and acero mostly just work (but slower obviously). > It's kind of fiddly in the arrow code but working, and means users can port > things easily. > > Once that is in, the plan is to submit a following pr that adds cmake > presets for emscripten which can build the cpp libraries and pyarrow for > pyodide. I've hacked this together in a build already, it's a bit fiddly > and needs a load of tidying up, but I'm confident it can be done. > > Essentially, I'm wanting to get this stuff in because pandas is moving > towards arrow as a pretty much required dependency, and webassembly is a > pandas platform, as well as > being an official python platform, so it would be great to get it working > in pyodide without us needing to maintain a load of patches. I guess it > could also come in handy with various container platforms that are moving > to webassembly. > > Basically I thought it's probably worth a bit of a heads up relating to > this, as I know the bigger picture of things is often hard to see from just > pull requests. > > Thanks > Joe > > > > This message and any attachment are intended solely for the addressee > and may contain confidential information. If you have received this > message in error, please contact the sender and delete the email and > attachment. > > Any views or opinions expressed by the author of this email do not > necessarily reflect the views of the University of Nottingham. Email > communications with the University of Nottingham may be monitored > where permitted by law. > > > > >
