vvcephei commented on a change in pull request #11582:
URL: https://github.com/apache/kafka/pull/11582#discussion_r771505654
##########
File path:
streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java
##########
@@ -186,6 +203,68 @@ public boolean setFlushListener(final
CacheFlushListener<K, V> listener,
return false;
}
+ @SuppressWarnings("unchecked")
+ @Override
+ public <R> QueryResult<R> query(final Query<R> query,
+ final PositionBound positionBound,
+ final boolean collectExecutionInfo) {
+
+ final long start = System.nanoTime();
+ final QueryResult<R> result;
+
+ final QueryHandler handler = queryHandlers.get(query.getClass());
+ if (handler == null) {
+ result = wrapped().query(query, positionBound,
collectExecutionInfo);
+ if (collectExecutionInfo) {
+ result.addExecutionInfo(
+ "Handled in " + getClass() + " in " + (System.nanoTime() -
start) + "ns");
+ }
+ } else {
+ result = (QueryResult<R>) handler.apply(
+ query,
+ positionBound,
+ collectExecutionInfo,
+ this
+ );
+ if (collectExecutionInfo) {
+ result.addExecutionInfo(
+ "Handled in " + getClass() + " with serdes "
+ + serdes + " in " + (System.nanoTime() - start) +
"ns");
+ }
+ }
+ return result;
+ }
+
+ @SuppressWarnings("unchecked")
+ private <R> QueryResult<R> runKeyQuery(final Query query,
+ final PositionBound positionBound, final boolean collectExecutionInfo)
{
+ final QueryResult<R> result;
+ final KeyQuery<K, V> typedQuery = (KeyQuery<K, V>) query;
+ final KeyQuery<Bytes, byte[]> rawKeyQuery =
KeyQuery.withKey(keyBytes(typedQuery.getKey()));
+ final QueryResult<byte[]> rawResult =
+ wrapped().query(rawKeyQuery, positionBound, collectExecutionInfo);
+ if (rawResult.isSuccess()) {
+ final boolean timestamped =
WrappedStateStore.isTimestamped(wrapped());
+ final Serde<V> vSerde = serdes.valueSerde();
+ final Deserializer<V> deserializer;
+ if (!timestamped && vSerde instanceof ValueAndTimestampSerde) {
Review comment:
Yes, this is super weird, and I think that
https://issues.apache.org/jira/browse/KAFKA-13526 will give us a more elegant
way to correct it, but as it stands right now, this is necessary.
The MeteredStore's serde is always a ValueAndTimestamp serde regardless of
whether the inner store is Timestamped or not. This works because the normal
execution flow actually converts the byte results from non-timestamped stores
into the binary schema of a ValueAndTimestamp.
What we do is, when you have a non-timestamped store, we wrap it with an
extra layer
(`org.apache.kafka.streams.state.internals.KeyValueToTimestampedKeyValueByteStoreAdapter`)
that pads the returned values with a fake timestamp
(`org.apache.kafka.streams.state.TimestampedBytesStore#convertToTimestampedFormat`).
That makes sense when the store is used by processors (particularly the ones
in the DSL) because it makes the store configuration orthogonal to the
processor logic, but for IQ, it's just spending extra time and memory for no
productive purpose.
One of the primary design goals of IQv2 is to make query execution as lean
as possible, so we did not implement the same padding logic for non-timestamped
data and instead just bubble up to the MeteredStore the actual byte array
returned from the BytesStore. Which means that if we want to deserialize it, we
need to know whether to use the ValueAndTimestamp deserializer or just the
Value's deserializer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
