steveloughran commented on code in PR #3481:
URL: https://github.com/apache/parquet-java/pull/3481#discussion_r3100426309
##########
parquet-variant/src/main/java/org/apache/parquet/variant/Variant.java:
##########
@@ -324,23 +333,64 @@ public int numArrayElements() {
* @throws IllegalArgumentException if `getType()` does not return
`Type.ARRAY`
*/
public Variant getElementAtIndex(int index) {
- VariantUtil.ArrayInfo info = VariantUtil.getArrayInfo(value);
+ VariantUtil.ArrayInfo info = arrayInfo();
if (index < 0 || index >= info.numElements) {
return null;
}
- return getElementAtIndex(
- index,
- value,
- metadata,
- info.offsetSize,
- value.position() + info.offsetStartOffset,
- value.position() + info.dataStartOffset);
+ int offsetStart = value.position() + info.offsetStartOffset;
+ int dataStart = value.position() + info.dataStartOffset;
+ int offset = VariantUtil.readUnsigned(value, offsetStart + info.offsetSize
* index, info.offsetSize);
+ return childVariant(VariantUtil.slice(value, dataStart + offset));
+ }
+
+ /**
+ * Creates a child Variant that shares this instance's metadata cache.
+ */
+ private Variant childVariant(ByteBuffer childValue) {
+ return new Variant(childValue, metadata, metadataCache, dictSize);
}
- private static Variant getElementAtIndex(
- int index, ByteBuffer value, ByteBuffer metadata, int offsetSize, int
offsetStart, int dataStart) {
- // offsetStart and dataStart are absolute positions in the `value` buffer.
- int offset = VariantUtil.readUnsigned(value, offsetStart + offsetSize *
index, offsetSize);
- return new Variant(VariantUtil.slice(value, dataStart + offset), metadata);
+ /**
+ * Returns the metadata dictionary string for the given ID, caching the
result.
+ */
+ String getMetadataKeyCached(int id) {
+ // Fall back to uncached lookup for out-of-range IDs
+ if (id < 0 || id >= dictSize) {
+ return VariantUtil.getMetadataKey(metadata, id);
+ }
+ // Demand-create shared dictionary cache
+ String[] cache = metadataCache;
+ if (cache == null) {
+ cache = new String[dictSize];
+ metadataCache = cache;
+ }
+ if (cache[id] == null) {
Review Comment:
set cache to be metadataCache here, so if there is any race condition the
same cache array is updated. If two threads are writing to the same [id], well,
one lookup is wasted. The joint cache is still (probably) updated with each
value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
