Github user PuspenduBanerjee commented on the pull request:
https://github.com/apache/nifi/pull/197#issuecomment-177686181
Hi @joewitt
For now the issue is happening for
org.apache.nifi.processors.standard.TestExecuteStreamCommand.testExecuteIngestAndUpdate().
Where we are running a Pattern on 100MB data. So, I have modified the
testcase with a sliding window matching mechanism. I hope it will work.:
/**
* Generates a String representation by taking a slice of backing
byte[] from MockFile.
* It's a low memory foot-print alternative to {@code new
String(mockFile.toByteArray)}
* @param mockFlowFile
* @param patternToMatch
* @return
* @throws NoSuchFieldException
* @throws SecurityException
* @throws IllegalArgumentException
* @throws IllegalAccessException
*/
private boolean matchPattern(MockFlowFile mockFlowFile, final String
patternToMatch) throws NoSuchFieldException, SecurityException,
IllegalArgumentException, IllegalAccessException {
final Pattern pattern= Pattern.compile(patternToMatch);
Field dataField=MockFlowFile.class.getDeclaredField("data");
dataField.setAccessible(true);
byte[] byteArray = (byte[])dataField.get(mockFlowFile);
final int
sliceLength=patternToMatch.length()>(10240)?patternToMatch.length():(10240);
final int windowLength=sliceLength-patternToMatch.length();
int offset=0;
StringBuilder stringBuilder=new StringBuilder();
while(offset<=byteArray.length){
if(pattern.matcher(stringBuilder.append(new
String(Arrays.copyOfRange(byteArray, offset, (offset+sliceLength))))).find()){
return true;
}
stringBuilder.delete(0, windowLength);
offset+=sliceLength;
};
return false;
}
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
