Hey Arrow Community, Do we have any work going on to produce/consume data from kafka and process it using arrow in python or any library involved in that. Just like we have a snowflake connector for python to read data super fast with the help of arrow.
Thanks , Mehul -----Original Message----- From: Antoine Pitrou <anto...@python.org> Sent: Wednesday, July 1, 2020 1:19 PM To: dev@arrow.apache.org Subject: Re: [Discuss] Extremely dubious Python equality semantics CAUTION: THIS EMAIL IS FROM AN EXTERNAL SOURCE. Internet links, office documents or other attachments may contain viruses. Do not click on a link, open or enable any file unless you trust the sender. Note if the snippet below doesn't display right in your e-mail reader, you can read it here: https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.github.com%2Fpitrou%2F6a0ce89ce866bc0c70e33155503d1c47&data=02%7C01%7CMehul.Batra%40pb.com%7Ce2a58aa5af3643767be608d81d932b55%7C8a4925a9fd8e4866b31cf719fb05dce6%7C0%7C0%7C637291865208746256&sdata=Xfkj9ga%2BGNAiEDuvYaJg6gaWZYoyPIjwP0U%2Fh0wS2bI%3D&reserved=0 Le 01/07/2020 à 09:46, Antoine Pitrou a écrit : > > Hello, > > Recent changes to PyArrow seem to have taken the stance that comparing > null values should return null. The problem is that it breaks the > expectation that comparisons should return booleans, and perculates > into crazy behaviour in other places. Here is an example of such > misbehaviour in the scalar refactor PR: > >>>> import pyarrow as pa > > >>>> na = pa.scalar(None) > > >>>> na == na > > > <pyarrow.NullScalar: None> >>>> na == 5 > > > <pyarrow.NullScalar: None> >>>> bool(na == 5) > > > True >>>> if na == 5: print("yo!") > > > yo! >>>> na in [5] > > > True > > But you can see it also with arrays containing null values: > >>>> pa.array([1, None]) in [pa.scalar(42)] > > > True > > I think that Python equality operators should behave in a > Python-sensible way (return True or False). Have people call another > method if they like the fancy (or noxious, depending on the POV) > semantics of returning null when comparing null with anything. > > (note that Numpy doesn't have null scalars, so it can be less > conservative in its customization of equality methods) > > Regards > > Antoine. >
