Here is additional food for thought. The cookbook currently contains examples for C++, R, and Python. Is there a plan (or wish) to eventually extend a single cookbook to include examples from other languages (eg., Rust, Java)? If so, then putting the cookbook into its own (language agnostic) repo would make more sense. On the other hand, if the cookbook is to be limited in Arrow languages, then what would happen if a Rust cookbook is created? Would it be placed in the arrow-rs repo or as a standalone arrow/cookbook-rs repo?
~Eduardo On Wed, Jul 7, 2021 at 8:09 PM Eduardo Ponce <edponc...@gmail.com> wrote: > Great work! > I would recommend having the cookbook in its own repo so that its updates > are not constrained by the timeline used for updating the public Arrow > documentation. > This will allow users that are not involved in Arrow development to > contribute or provide suggestions to the cookbook fairly easily (no need to > download Arrow and build doc). > Also, the repo could be used to provide example programs for some of the > cookbook's recipes and even have a place for users to share their own > examples. > > ~Eduardo > > On Wed, Jul 7, 2021 at 7:45 PM Rares Vernica <rvern...@gmail.com> wrote: > >> Awesome! We would find C++ versions of these recipes very useful. From our >> experience the C++ API is much much harder to deal with and error prone >> than the R/Python one. >> >> Cheers, >> Rares >> >> On Wed, Jul 7, 2021 at 9:07 AM Alessandro Molina < >> alessan...@ursacomputing.com> wrote: >> >> > Yes, that was mostly what I meant when I wrote that the next step is >> > opening a PR against the apache/arrow repository itself :D >> > We moved forward in a separate repository initially to be able to cycle >> > more quickly, but we reached a point where we think we can start >> > integrating the cookbook with the Arrow documentation itself. >> > >> > If instead it's preferred to move forward the effort into its own >> separated >> > repository (apache/arrow-cookbook) that's an option too, we are open to >> > suggestions from the community. >> > >> > On Wed, Jul 7, 2021 at 5:57 PM Wes McKinney <wesmck...@gmail.com> >> wrote: >> > >> > > What do you think about developing this cookbook in an Apache Arrow >> > > repository (it could be something like apache/arrow-cookbook, if not >> > > part of the main development repo)? Creating expanded documentation >> > > resources for learning how to use Apache Arrow to solve problems seems >> > > certainly within the bounds of the community's objectives. >> > > >> > > On Wed, Jul 7, 2021 at 5:52 PM Alessandro Molina >> > > <alessan...@ursacomputing.com> wrote: >> > > > >> > > > We finally have a first preview of the cookbook available for R and >> > > Python, >> > > > for anyone interested the two versions are visible at >> > > > http://ursacomputing.com/arrow-cookbook/py/index.html and >> > > > http://ursacomputing.com/arrow-cookbook/r/index.html >> > > > A new version of the cookbook is automatically published on each new >> > > recipe. >> > > > >> > > > After gathering feedback from interested parties and users, our plan >> > for >> > > > the next step would be to open a PR against the arrow repository and >> > > > automate publishing the cookbook via github actions. >> > > > >> > > > At the moment the recipes implemented are nearly half of those that >> > were >> > > > identified in the dedicated Google Docs ( >> > > > >> > > >> > >> https://docs.google.com/document/d/1v-jK_9osnLvAnAjLOM_frgzakjFhLpUi8OC0MlKpxzw/edit?ts=60c73189#heading=h.m7fas2talgy5 >> > > > ) so if you have recipes to suggest feel free to leave comments on >> that >> > > > document or suggest edits. >> > > > >> > > > >> > > > On Mon, Jun 21, 2021 at 10:34 AM Alessandro Molina < >> > > > alessan...@ursacomputing.com> wrote: >> > > > >> > > > > Hi, >> > > > > >> > > > > I'd like to share with the ML an idea which me and Nic Crane have >> > been >> > > > > experimenting with. It's still in the early stage, but we hope to >> > turn >> > > it >> > > > > into a PR for Arrow documentation soon. >> > > > > >> > > > > The idea is to work on a Cookbook, a collection of ready made >> > recipes, >> > > on >> > > > > how to use Arrow that both end users and developers of third party >> > > > > libraries can refer to when they need to look up "the arrow way" >> of >> > > doing >> > > > > something. >> > > > > >> > > > > While the arrow documentation reports all features and functions >> that >> > > are >> > > > > available in arrow, it's not always obvious how to best combine >> them >> > > for a >> > > > > new user. Sometimes the solution ends up being more complicated >> than >> > > > > necessary or performs badly due to not obvious side effects like >> > > unexpected >> > > > > memory copies etc. >> > > > > >> > > > > For this reason we thought about starting a documentation that >> users >> > > can >> > > > > refer to on how to combine arrow features to achieve the results >> they >> > > care >> > > > > about. >> > > > > >> > > > > We wrote a short document explaining the idea at >> > > > > >> > > >> > >> https://docs.google.com/document/d/1v-jK_9osnLvAnAjLOM_frgzakjFhLpUi8OC0MlKpxzw/edit?usp=sharing >> > > > > >> > > > > The core idea behind the cookbook is that all recipes should be >> > > testable, >> > > > > so it should be possible to add a CI phase for the cookbook that >> > > verifies >> > > > > that all the recipes still work with the current version of Arrow >> and >> > > lead >> > > > > to the expected results. >> > > > > >> > > > > At the moment we started it in a separate repository ( >> > > > > https://github.com/ursacomputing/arrow-cookbook ), but we are yet >> > > unsure >> > > > > if it should live inside arrow/docs or its own directory (IE: >> > > > > arrow/cookbook) or its own repository. In the end it's fairly >> > decoupled >> > > > > from the rest of Arrow and the documentation, which would have the >> > > benefit >> > > > > of allowing a dedicated release cycle every time new recipes are >> > added >> > > (at >> > > > > least in the early phase). >> > > > > >> > > > > We are also looking for more ideas about recipes that would be >> good >> > > > > candidates for inclusion, so if any of you has thoughts about >> which >> > > recipes >> > > > > we should add please feel free to comment on the document or >> reply by >> > > mail >> > > > > suggesting more recipes. >> > > > > >> > > > > Any suggestion for improvements is appreciated! We hope to have >> > > something >> > > > > we can release with the next Arrow release. >> > > > > >> > > >> > >> >
