voonhous commented on code in PR #19707:
URL: https://github.com/apache/hudi/pull/19707#discussion_r3833032950
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/sources/helpers/KinesisDeaggregator.java:
##########
@@ -44,36 +59,137 @@ public static List<Record> deaggregate(List<Record>
records) {
if (records == null || records.isEmpty()) {
return new ArrayList<>();
}
- List<com.amazonaws.services.kinesis.model.Record> v1Records = new
ArrayList<>(records.size());
- for (Record r : records) {
- v1Records.add(toV1Record(r));
- }
- List<UserRecord> userRecords = UserRecord.deaggregate(v1Records);
- List<Record> result = new ArrayList<>(userRecords.size());
- for (UserRecord ur : userRecords) {
- result.add(toV2Record(ur));
+ List<Record> result = new ArrayList<>(records.size());
+ for (Record record : records) {
+ byte[] data = record.data() == null ? null : record.data().asByteArray();
Review Comment:
Switched to `asByteArrayUnsafe()`, with a comment noting the array is only
read and sub-record payloads are copied out by `readByteArray()`.
##########
pom.xml:
##########
@@ -812,18 +810,6 @@
<dependencyManagement>
<dependencies>
Review Comment:
Added both to `bannedDependencies`. This bans only the ASL 1.x line: KCL 2.x
moved to `software.amazon.kinesis`, so an Apache-licensed KCL stays usable if
ever needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
