Hi,
ArrayKernelExec must be a pointer to a C function.
using ArrayKernelExec = Status (*)(KernelContext*, const ExecSpan&,
ExecResult*);
Status EncryptFloat64(KernelContext* ctx, const ExecSpan& batch,
ExecResult* out) {
auto& arg0 = batch[0];
auto out_data = PrealocateBinaryArrayForMyEncryption(arg0,
/*source_data_size*/sizeof(double));
if (arg0.MayHaveNulls()) {
// specialized loop encrypting all the values (handling nulls)
} else {
// specialized loop encrypting all the values (ignoring validity buffer)
}
*out = std::move(out_data);
return Status::OK();
}
const arrow::Status encrstatus = encryptfunc.AddKernel({arrow::float64()},
arrow::binary(), EncryptFloat64);
__
Felipe
On Sun, Jul 7, 2024 at 5:51 PM Prateem Mandal <[email protected]> wrote:
> Hello,
>
> I am implementing an encryption and decryption transformation function
> that takes a parquet file as input and encrypts each column of each row
> using AES-CTR. (I am aware of Parquet Modular Encryption but that is not an
> option right now for various reasons).
>
> I am using Arrow to read, process and write encrypted/decrypted files. I
> am doing this in C++ due to performance reasons.
>
> I was able to read the parquet file, recordbatch by record batch and for
> each record batch, I am attempting to apply encryption/decryption to each
> column array using my own provided compute function. I am following the
> pattern shown here
> <https://arrow.apache.org/docs/cpp/tutorials/compute_tutorial.html#calculating-element-wise-array-addition-with-callfunction>
> .
>
> In order to proceed I need to add my encryption/decryption function. I am
> doing something like the following
>
>
> *arrow::compute::ScalarFunction encryptfunc =
> arrow::compute::ScalarFunction("encr", arrow::compute::Arity::Unary(),
> arrow::compute::FunctionDoc::Empty());*
> *const arrow::Status &encrstatus =
> encryptfunc.AddKernel({arrow::float64()}, arrow::binary(),
> arrow::compute::ArrayKernelExec(???));*
>
> Now I am unsure where I can plug in my lambda function that encapsulates
> the key and encryption logic? Is this even possible?
>
> Thanks
> Prateem
>
>
