[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427331#comment-16427331
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2497


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
> Fix For: 1.7.0
>
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427333#comment-16427333
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2497
  
@mosermw Looks great! Thanks, this has been merged to master.


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
> Fix For: 1.7.0
>
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16427329#comment-16427329
 ] 

ASF subversion and git services commented on NIFI-3599:
---

Commit dc9b4cb516da346a2942d0bef2810b60ac179cc7 in nifi's branch 
refs/heads/master from [~boardm26]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=dc9b4cb ]

NIFI-3599 Allow back pressure object count and data size to be configurable in 
nifi.properties. This closes #2497


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424631#comment-16424631
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mosermw commented on the issue:

https://github.com/apache/nifi/pull/2497
  
Hi @mcgilman I made the suggested changes, rebased from master, and force 
pushed to this branch. What do you think?


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402425#comment-16402425
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2497#discussion_r175200879
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ConfigDTO.java
 ---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api.dto;
+
+import io.swagger.annotations.ApiModelProperty;
+
+import javax.xml.bind.annotation.XmlType;
+
+/**
+ * Contains configuration information for this NiFi.
+ */
+@XmlType(name = "config")
+public class ConfigDTO {
--- End diff --

Sorry for the earlier misguidance but it turns that there is already is a 
config dto. Its call `FlowConfigurationDTO` and is already query when the UI is 
loaded. We are already using this other endpoint for returning configuration 
information for this NiFi instance.


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402427#comment-16402427
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2497#discussion_r175197796
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardFlowFileQueue.java
 ---
@@ -139,6 +137,36 @@ public StandardFlowFileQueue(final String identifier, 
final Connection connectio
 
 readLock = new TimedLock(this.lock.readLock(), identifier + " Read 
Lock", 100);
 writeLock = new TimedLock(this.lock.writeLock(), identifier + " 
Write Lock", 100);
+
+NiFiProperties nifiProps = 
NiFiProperties.createBasicNiFiProperties(null, null);
--- End diff --

This current approach will load the properties file every time a connection 
is created. Can we follow the approach that was taken with the 
`queueSwapThreshold` instead? Usually, this sort of parsing logic is in 
`NiFiProperties` so that `getBackPressureObjectThreshold` and 
`getBackPressureDataSizeThreshold` return the parsed value. 


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402424#comment-16402424
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2497#discussion_r175198349
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-connection-configuration.js
 ---
@@ -83,6 +84,34 @@
 });
 }
 
+/**
+ * Loads the controller configuration.
+ */
+var loadBackPressureDefaults = function () {
+var deferred = $.Deferred();
+$.ajax({
--- End diff --

The ajax result is a deferred. In this case, you simply return it. For 
instance `return $.ajax({...});`


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402428#comment-16402428
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2497#discussion_r175201659
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-connection-configuration.js
 ---
@@ -1272,11 +1301,9 @@
 }
 
 // initialize the connection dialog
-$.when(initializeSourceNewConnectionDialog(source), 
initializeDestinationNewConnectionDialog(destination)).done(function () {
+$.when(initializeSourceNewConnectionDialog(source), 
initializeDestinationNewConnectionDialog(destination), 
loadBackPressureDefaults()).done(function () {
--- End diff --

This approach is going to query for the backpressure defaults every time a 
new connection is opened. Since these values are not changing, we just need to 
load them once. If we end up using the existing /nifi-api/flow/config endpoint, 
we won't need to query for these here at all. Instead, the default values can 
be passed into the `nfConnectionConfiguration.init(...)` function. This 
function is invoked inside of the response handler of the flow configuration 
endpoint. Inside of the `init(...)` function we could just hold a reference to 
these values that could be restored every time the new connection dialog is 
opened. 


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402426#comment-16402426
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2497#discussion_r175195219
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ConfigDTO.java
 ---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api.dto;
+
+import io.swagger.annotations.ApiModelProperty;
+
+import javax.xml.bind.annotation.XmlType;
+
+/**
+ * Contains configuration information for this NiFi.
+ */
+@XmlType(name = "config")
+public class ConfigDTO {
+
+private String backPressureObjectThreshold;
+private String backPressureDataSizeThreshold;
--- End diff --

Should we rename these `defaultBackPressureObjectThreshold` and  
`defaultBackPressureDataSizeThreshold`? Also, we can make the 
`defaultBackPressureObjectThreshold` a `Long`.


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401963#comment-16401963
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2497
  
@mosermw Didn't really have a great option... so the suggestion was 
ultimately a compromise. The idea was that About returns information about this 
instance of NiFi. This could include some configuration like the 
defaultBackpressThreshold. We could eventually move other things like the 
content viewer URL in there too. 


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401115#comment-16401115
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mosermw commented on the issue:

https://github.com/apache/nifi/pull/2497
  
@mcgilma @markap14 I'm not sure I was able to fully grok "add a ConfigDTO 
which is set on the AboutDTO" but I gave it a shot.  Rebased from master and 
pushed a new commit.  Let me know what you think, thanks!


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395645#comment-16395645
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/2497
  
@mosermw @markap14 It's kind of a gray area. I suggested an endpoint like 
the /nifi-api/flow/about because that seemed like the best fit currently and 
was obviously trying to avoid requiring another request to load the UI. I'm a 
little hesitant to go the route of a /nifi-api/flow/properties endpoint because 
I'm not sure that's a concept we want to advertise/expose. I don't mind doing 
something more generic, but what if the concept was more related to default 
values or config. This is something that any client using the API may want to 
know ahead of time for this reason exactly. Would adding a DefaultsDTO or 
ConfigDTO which is set on the AboutDTO fit a little better? 


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386508#comment-16386508
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user mosermw commented on the issue:

https://github.com/apache/nifi/pull/2497
  
@markap14 and @mcgilman I did consider that backpressure settings didn't 
really belong in AboutDTO. The BannerDTO also pulls information from 
nifi.properties, but I didn't think backpressure fit there either.  I didn't 
want to further expand the API by adding something like BackpressureDTO at a 
/nifi-api/flow/backpressure endpoint, but maybe that's the preferred approach? 
Or perhaps a PropertiesDTO at /nifi-api/flow/properties to do something more 
generic?


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383596#comment-16383596
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2497
  
@mosermw I think this is a pretty reasonable change. The only thing that 
gives me pause is putting this information into AboutDTO. To me, it doesn't 
feel like the right place for this kind of information. Admittedly, I don't 
know off the top of my head where would be the best place for it. @mcgilman do 
you have any thoughts on where something like that should go?


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378812#comment-16378812
 ] 

ASF GitHub Bot commented on NIFI-3599:
--

GitHub user mosermw opened a pull request:

https://github.com/apache/nifi/pull/2497

NIFI-3599 Allowed back pressure object count and data size to be 
configurable

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mosermw/nifi nifi-3599

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2497.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2497


commit 5ba57cb9c10008b173e5b928979c96e9ade2f0b5
Author: Mike Moser 
Date:   2018-02-26T19:04:04Z

NIFI-3599 Allowed back pressure object count and data size to be 
configurable




> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Michael Moser
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[Michael Moser (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374892#comment-16374892
 ] 

Michael Moser commented on NIFI-3599:
-

Thank you both for the feedback.  I agree showing the actual values in those 
fields is best, and I'll look into modifying the /nifi-api/flow/about endpoint 
response to provide them.

> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Jeremy Dyer
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[Matt Gilman (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374828#comment-16374828
 ] 

Matt Gilman commented on NIFI-3599:
---

[~mosermw] We currently have an about endpoint that we use to relay server-side 
information to the front end (content viewer uri, version, timezone, etc). Some 
of these values come from nifi.properties. I would suggest adding to an 
endpoint like this so that the user knows that the default actual is and 
whether that's an appropriate value for the connection. Additionally, by still 
populating an actual value in the UI we are guaranteed that the same value will 
be applied across the cluster. If we allowed the user to specify a blank value 
or 'default' alias then if nodes have different defaults, we would have 
inconsistent flow configurations.

> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Jeremy Dyer
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[Scott Aslan (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374824#comment-16374824
 ] 

Scott Aslan commented on NIFI-3599:
---

You are correct that the UI does not have access to nifi.properties but I will 
let [~mcgilman] comment as to whether this approach makes sense on the server 
side. As for the UX, instead of leaving the field blank we should disable it 
and have a checked checkbox next to that field indicating that the 'default' 
values from nifi.properties will be used. Then if the user wants to override 
those values they would uncheck the checkbox and enter the new back pressure 
threshold values...

> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Jeremy Dyer
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[Michael Moser (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374737#comment-16374737
 ] 

Michael Moser commented on NIFI-3599:
-

I have an approach to resolve this, but I would like to get [~mcgilman] and/or 
[~scottyaslan] to comment, because there are UI/UX implications.

It's fairly easy to move default back pressure Object and Data Size threshold 
settings from server-side code (StandardFlowFileQueue.java) to nifi.properties 
and make the back end use them.  However, the UI also has default back pressure 
set in the nf-connection-configuration.js code.  The UI does not seem to have 
access to nifi.properties in order to read settings from there.

When a new connection is drawn, I propose setting these two back pressure 
fields to 'default' in the UI, or leave them empty.  If a user doesn't change 
them, the JS would send to the server a null value in the JSON for these two 
fields.  The server would recognize this and use the nifi.properties default 
back pressure settings.  If a user makes changes to these fields, the JSON sent 
to the server would contain those changes.

I tested this approach and it works.  I'll be happy to submit a PR.  But is 
this an acceptable approach?  Thanks for feedback.

> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Jeremy Dyer
>Priority: Major
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-3599) Add nifi.properties value to globally set the default backpressure size threshold for each connection

[Toivo Adams (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962687#comment-15962687
 ] 

Toivo Adams commented on NIFI-3599:
---

It would be very useful to be able set both 
Back Pressure Object Threshold
and Back Pressure Data Size Threshold
default values in  nifi.properties

[~jeremy.dyer], do you have a time to implement this?

Thanks
Toivo


> Add nifi.properties value to globally set the default backpressure size 
> threshold for each connection
> -
>
> Key: NIFI-3599
> URL: https://issues.apache.org/jira/browse/NIFI-3599
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Jeremy Dyer
>Assignee: Jeremy Dyer
>
> By default each new connection added to the workflow canvas will have a 
> default backpressure size threshold of 10,000 objects. While the threshold 
> can be changed on a connection level it would be convenient to have a global 
> mechanism for setting that value to something other than 10,000. This 
> enhancement would add a property to nifi.properties that would allow for this 
> threshold to be set globally unless otherwise overridden at the connection 
> level.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)