cgivre commented on code in PR #2580:
URL: https://github.com/apache/drill/pull/2580#discussion_r908416571
##########
contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaGroupScan.java:
##########
@@ -262,6 +269,17 @@ public void applyAssignments(List<DrillbitEndpoint>
incomingEndpoints) {
assignments = AssignmentCreator.getMappings(incomingEndpoints,
Lists.newArrayList(partitionWorkMap.values()));
}
+ @Override
+ public GroupScan applyLimit(int maxRecords) {
+ records = maxRecords; // Just apply the limit value into sub-scan
+ return super.applyLimit(maxRecords);
+ }
+
Review Comment:
I'm fairly certain this isn't actually working. The super method always
returns `null`.
https://github.com/apache/drill/blob/53e6f2697b31cb76a38e75fe95283512715d28c3/exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScan.java#L174-L178
What I think you need to do here is:
1. Create a clone constructor in the `KafkaGroupScan`
2. Call that and return a new `KafkaGroupScan` with the limit applied.
##########
contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaGroupScan.java:
##########
@@ -77,6 +77,7 @@ public class KafkaGroupScan extends AbstractGroupScan {
private final KafkaScanSpec kafkaScanSpec;
private List<SchemaPath> columns;
+ private int records;
Review Comment:
These variables should all be `final`.
##########
contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaGroupScan.java:
##########
@@ -359,7 +382,7 @@ public KafkaStoragePlugin getStoragePlugin() {
@Override
public String toString() {
- return String.format("KafkaGroupScan [KafkaScanSpec=%s, columns=%s]",
kafkaScanSpec, columns);
+ return String.format("KafkaGroupScan [KafkaScanSpec=%s, columns=%s,
records=%d]", kafkaScanSpec, columns, records);
Review Comment:
Could we use the `PlanStringBuilder` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
