Re: [Python] Any reason to exclude __lt__ from ArrayValue ?

[Uwe L. Korn]

Hello John,

as with most things concering the *Value classes: Missing implementations are 
simply "not-done-yet" and not explicit omissions. The value instances have not 
yet seen that much use and therefore lack a lot of functionality. Feel free to 
add this to them.

Uwe

On Sat, May 25, 2019, at 6:01 AM, John Muehlhausen wrote:
> We have __eq__ leaning on as_py() already ... any reason not to have __lt__
> ?
> 
> This makes it possible to use bisect to find slices in ordered data without
> a __getitem__ wrapper:
> 
> 1176.0  key=pa.array(['AAPL'])
>  110.0  print(bisect.bisect_left(batch[3],key[0]))
>   64.0  print(bisect.bisect_right(batch[3],key[0]))
> 
> Although, I'm not sure why pa.array() is relatively slow (above in mics)
> and whether I can directly construct an individual Value instead?  batch[3]
> is string type with length 32291182 (memory mapped from IPC File)... AAPL
> being slice 206424 to 420255 in this case.
> 
> Proposed addition:
> 
> def __eq__(self, other):
> if hasattr(self, 'as_py'):
> if isinstance(other, ArrayValue):
> other = other.as_py()
> return self.as_py() == other
> else:
> raise NotImplementedError(
> "Cannot compare Arrow values that don't support as_py()")
> 
> *def __lt__(self, other):*
> *if hasattr(self, 'as_py'):*
> *if isinstance(other, ArrayValue):*
> *other = other.as_py()*
> *return self.as_py() < other*
> *else:*
> *raise NotImplementedError(*
> *"Cannot compare Arrow values that don't support as_py()")*
>

Re: Python development setup and LLVM 7 / Gandiva

[Uwe L. Korn]

Hello John,

I guess you also have some other llvm-* packages installed on OSX. We currently 
have the problem that they override each other on OSX: 
https://github.com/conda-forge/llvmdev-feedstock/issues/60 The compilers 
shipped by conda-forge on OSX use llvm=4.0.1 and thus this is also installed at 
the same time.

Uwe

On Thu, May 23, 2019, at 9:31 PM, John Muehlhausen wrote:
> Not sure why cmake isn't happy (as in original post).  Environment is set
> up as per instructions:
> 
> (pyarrow-dev) JGM-KTG-Mac-Mini:python jmuehlhausen$ conda list llvmdev
> # packages in environment at
> /Users/jmuehlhausen/miniconda3/envs/pyarrow-dev:
> #
> # NameVersion   Build  Channel
> llvmdev   7.0.0 h04f5b5a_1000conda-forge
> 
> On Thu, May 23, 2019 at 1:46 PM Wes McKinney  wrote:
> 
> > llvmdev=7 is in the conda_env_cpp.yml requirements file, are you using
> > something else?
> >
> > https://github.com/apache/arrow/blob/master/ci/conda_env_cpp.yml#L31
> >
> > On Thu, May 23, 2019 at 12:53 PM John Muehlhausen  wrote:
> > >
> > > The pyarrow-dev conda environment does not include llvm 7, which appears
> > to
> > > be a requirement for Gandiva.
> > >
> > > So I'm just trying to figure out a pain-free way to add llvm 7 in a way
> > > that cmake can find it, for Mac.
> > >
> > > I had already solved the other Mac problem with
> > > export CONDA_BUILD_SYSROOT=/Users/jmuehlhausen/sdks/MacOSX10.9.sdk
> > >
> > > On Wed, May 22, 2019 at 1:46 PM Wes McKinney 
> > wrote:
> > >
> > > > hi John,
> > > >
> > > > Some changes were just made to address the issue you are having, see
> > > > the latest instructions at
> > > >
> > > >
> > > >
> > https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst
> > > >
> > > > Let us know if that does not work.
> > > >
> > > > - Wes
> > > >
> > > > On Wed, May 22, 2019 at 11:02 AM John Muehlhausen  wrote:
> > > > >
> > > > > Set up pyarrow-dev conda environment as at
> > > > > https://arrow.apache.org/docs/developers/python.html
> > > > >
> > > > > Got the following error.  I will disable Gandiva for now but I'd
> > like to
> > > > > get it back at some point.  I'm on Mac OS 10.13.6.
> > > > >
> > > > > CMake Error at cmake_modules/FindLLVM.cmake:33 (find_package):
> > > > >   Could not find a configuration file for package "LLVM" that is
> > > > compatible
> > > > >   with requested version "7.0".
> > > > >
> > > > >   The following configuration files were considered but not accepted:
> > > > >
> > > > >
> > > > >
> > > >
> > /Users/jmuehlhausen/miniconda3/envs/pyarrow-dev/lib/cmake/llvm/LLVMConfig.cmake,
> > > > > version: 4.0.1
> > > > >
> > > > >
> > > >
> > /Users/jmuehlhausen/miniconda3/envs/pyarrow-dev/lib/cmake/llvm/llvm-config.cmake,
> > > > > version: unknown
> > > > >
> > > > > Call Stack (most recent call first):
> > > > >   src/gandiva/CMakeLists.txt:31 (find_package)
> > > >
> >
>

[jira] [Created] (ARROW-5420) Implement or remove getCurrentSizeInBytes in VariableWidthVector

[Ji Liu (JIRA)]

Ji Liu created ARROW-5420:
-

 Summary: Implement or remove getCurrentSizeInBytes in 
VariableWidthVector
 Key: ARROW-5420
 URL: https://issues.apache.org/jira/browse/ARROW-5420
 Project: Apache Arrow
  Issue Type: Improvement
Reporter: Ji Liu
Assignee: Ji Liu


Now VariableWidthVector#getCurrentSizeInBytes doesn't seem to have been 
implemented. We should implement it or just remove it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)