sgilmore10 opened a new pull request, #38530:
URL: https://github.com/apache/arrow/pull/38530
<!--
Thanks for opening a pull request!
If this is your first pull request you can find detailed information on how
to contribute here:
* [New Contributor's
Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
* [Contributing
Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
If this is not a [minor
PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes).
Could you open an issue for this pull request on GitHub?
https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the
[Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.)
of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
In the case of PARQUET issues on JIRA the title also supports:
PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
-->
### Rationale for this change
Adding this `ClassTypeValidator` class is a step towards implementing the
`arrow.array.ListArray.fromMATLAB()` method for creating `ListArray`s whose
`ValueType`s is either a numeric, boolean, string, time32, or time64 array from
a MATLAB `cell` array.
### What changes are included in this PR?
Added an abstract class `arrow.array.internal.list.ListTypeValidator` that
defines three abstract methods:
1. `validateElement(obj, element)`
2. `length = getElementLength(obj, element)`
3. `C = reshapeCellElements(obj, C)`
These abstract methods will be used in `ListArray::fromMATLAB` to create
`ListArray`s from MATLAB `cell` arrays. Below is "pared-down" version of how
`fromMATLAB`'s algorithm:
```matlab
function listArray = fromMATLAB(C)
% Create the appropriate ListTypeValidator from the
% first element in the cell array C
validator = createListTypeValidator(C{1});
% Pre-allocate a uint32 vector for the offsets
numRows = numel(C);
offsets = zeros([numRows 1], "int32");
for ii = 1:numRows
cellElement = C{ii};
% Validate cellElement can be used to create
% one row in the ListArray, i.e. For example,
% if the first element in C was a double, verify
% cellElement is also a double.
validator.validateElement(cellElement);
% Determine how much to increment the
% last offset value by to set the offset at index ii + 1.
length = validator.getElementLEngth(cellElement);
offsets[ii + 1] = length + offsets[i];
end
% Reshape the elements in cell array C so that they
% can be vertically concatenated.
C = validator.reshapeCellElements(C);
% Extract the cell array elements and vertically concatenate
% them into one array. Then pass this array to arrow.array().
values = vertcat(C{:});
valueArray = arrow.array(values);
% Create an int32Array from offsets
offetArray = arrow.array(offsets);
listArray = arrow.array.ListArray(Values=valueArray, Offsets=offsetArray)
end
```
Depending on the `validator` object is created based on the first element in
the `cell` array `C`. We use the first element to determine what kind of
`ListArray` to construct from the input `cell` array.
--
Added a concrete class called
`arrow.array.internal.list.ClassTypeValidator`, which implements
`arrow.array.internal.list.ListTypeValdiator`:
1. `validateElement(obj, element)` - Throws an error if the element's class
type does not match the expected value.
2. `length = getElementLength(obj, element)` - Returns the number of
elements in the input array.
3. `C = reshapeCellElements(obj, C)` - Reshapes all elements in the `cell`
array `C` to be column vectors.
`ClassTypeValidator` will be used when creating `ListArray`s from MATLAB
`cell` arrays containing "primitive types", such as numerics, strings, and
durations.
### Are these changes tested?
Yes. I added a new class called `tClassTypeValidator.m`.
### Are there any user-facing changes?
No.
### Future Directions
1. #38420
2. #38417
3. #38354
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
