joewitt commented on pull request #5324: URL: https://github.com/apache/nifi/pull/5324#issuecomment-1033295071
I agree those are excellent examples of inconsistency and we should ensure we have disclaimers on them about usage. In a nifi 2.0 world I think we ought to punt those out for the reasons we are talking about here or at least make the danger of their usage more obvious. I'm not sure that view is widely shared but the more I spend time with users who truly just dont know these things the more we have to find reasonable ways to provide safety. On the other hand...the more we turn this into a safety tool the less powerful it is for the more capable people. I'm not sure what the best way to thread that needle is without coming across or acting inconsistently. Maybe we were on the right track with annotations which alerted the user to such things but we need to make that a more prominent part of the user experience. otto i do think we have some limit on the size of attributes or at least I know we considered it. On the other hand we also have to consider how many attributes there can be on a given flowfile. But then on top of that we have to consider how many flowfiles can be in the system at any one time and of that how many can have their metadata in memory at any one time. It is not uncommon to see real user flows with hundreds of thousands or millions of flowfiles in flight at any one time under normal conditions. The swapping stuff helps but doesn't eliminate the problem. Hard limits become tricky as they're fairly arbitrary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
