tibrewalpratik17 commented on code in PR #13103:
URL: https://github.com/apache/pinot/pull/13103#discussion_r1617789693
##########
pinot-spi/src/main/java/org/apache/pinot/spi/data/FieldSpec.java:
##########
@@ -98,6 +100,10 @@ public abstract class FieldSpec implements
Comparable<FieldSpec>, Serializable {
}
}
+ public enum MaxLengthExceedStrategy {
+ TRIM_LENGTH, FAIL_INGESTION, SUBSTITUTE_DEFAULT_VALUE, NO_ACTION
Review Comment:
`continueOnError` doesn't affect this Sanitization flow at all right? Even
if continueOnError is true, we would still end up skipping the record if we
throw exception from SanitizationTransformer.
```
try {
_recordEnricherPipeline.run(decodedRow.getResult());
_transformPipeline.processRow(decodedRow.getResult(),
reusedResult);
} catch (Exception e) {
_numRowsErrored++;
// when exception happens we prefer abandoning the whole batch and
not partially indexing some rows
reusedResult.getTransformedRows().clear();
String errorMessage =
String.format("Caught exception while transforming the record
at offset: %s , row: %s", offset,
decodedRow.getResult());
_segmentLogger.error(errorMessage, e);
_realtimeTableDataManager.addSegmentError(_segmentNameStr, new
SegmentErrorInfo(now(), errorMessage, e));
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
