[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-17 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2051


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-10 Thread markap14 
Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132502384
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-10 Thread markap14 
Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132477480
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-10 Thread markap14 
Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132476153
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-10 Thread markap14 
Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132472005
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -314,7 +430,7 @@ public Response updateProcessGroup(
 Authorizable authorizable = 
lookup.getProcessGroup(id).getAuthorizable();
 authorizable.authorize(authorizer, 
RequestAction.WRITE, NiFiUserUtils.getNiFiUser());
 },
-null,
+() -> 
serviceFacade.verifyUpdateProcessGroup(requestProcessGroupDTO),
--- End diff --

Good call. Originally, I had it updating the variables here but then 
refactored quite a bit. I think it's best to go ahead and leave in the addition 
of the verifyUpdateProcessGroup method, but make the verification a NOP. We do 
this in a few other places, as well, so I feel it's best to leave the 
'plumbing' there, since it's already been built. It will make it easier to 
update later.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-09 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132257748
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -238,6 +313,47 @@ public Response getProcessGroup(
 return generateOkResponse(entity).build();
 }
 
+
+/**
+ * Retrieves the Variable Registry for the group with the given ID
+ *
+ * @param groupId the ID of the Process Group
+ * @return the Variable Registry for the group
+ */
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{id}/variable-registry")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistry(
--- End diff --

Can we add a flag (probably enabled by default) that will include all 
variables available in this scope (defined in this Process Group and any 
ancestors)? Also, the response should indicate where each variable is defined.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-09 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132257197
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessGroupDTO.java
 ---
@@ -200,4 +204,13 @@ public void setInactiveRemotePortCount(Integer 
inactiveRemotePortCount) {
 this.inactiveRemotePortCount = inactiveRemotePortCount;
 }
 
+
+@ApiModelProperty("The variables that are configured for the Process 
Group")
+public Map getVariables() {
+return variables;
+}
+
+public void setVariables(final Map variables) {
--- End diff --

Also, can we update the documentation to indicate that this only includes 
the variables defined at this Process Group?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131985519
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardProcessGroupDAO.java
 ---
@@ -99,29 +116,69 @@ public void verifyScheduleComponents(final String 
groupId, final ScheduledState
 }
 
 @Override
-public void scheduleComponents(final String groupId, final 
ScheduledState state, final Set componentIds) {
+public void verifyActivateControllerServices(final String groupId, 
final ControllerServiceState state, final Set serviceIds) {
+final ProcessGroup group = locateProcessGroup(flowController, 
groupId);
+
+group.findAllControllerServices().stream()
+.filter(service -> 
serviceIds.contains(service.getIdentifier()))
+.forEach(service -> {
+if (state == ControllerServiceState.ENABLED) {
+service.verifyCanEnable();
+} else {
+service.verifyCanDisable();
+}
+});
+}
+
+@Override
+public CompletableFuture scheduleComponents(final String 
groupId, final ScheduledState state, final Set componentIds) {
 final ProcessGroup group = locateProcessGroup(flowController, 
groupId);
 
+CompletableFuture future = 
CompletableFuture.completedFuture(null);
+
 for (final String componentId : componentIds) {
 final Connectable connectable = 
group.findLocalConnectable(componentId);
 if (ScheduledState.RUNNING.equals(state)) {
 if 
(ConnectableType.PROCESSOR.equals(connectable.getConnectableType())) {
-
connectable.getProcessGroup().startProcessor((ProcessorNode) connectable);
+final CompletableFuture processorFuture = 
connectable.getProcessGroup().startProcessor((ProcessorNode) connectable);
+future = CompletableFuture.allOf(future, 
processorFuture);
 } else if 
(ConnectableType.INPUT_PORT.equals(connectable.getConnectableType())) {
 connectable.getProcessGroup().startInputPort((Port) 
connectable);
 } else if 
(ConnectableType.OUTPUT_PORT.equals(connectable.getConnectableType())) {
 connectable.getProcessGroup().startOutputPort((Port) 
connectable);
 }
 } else {
 if 
(ConnectableType.PROCESSOR.equals(connectable.getConnectableType())) {
-
connectable.getProcessGroup().stopProcessor((ProcessorNode) connectable);
+final CompletableFuture processorFuture = 
connectable.getProcessGroup().stopProcessor((ProcessorNode) connectable);
+future = CompletableFuture.allOf(future, 
processorFuture);
 } else if 
(ConnectableType.INPUT_PORT.equals(connectable.getConnectableType())) {
 connectable.getProcessGroup().stopInputPort((Port) 
connectable);
 } else if 
(ConnectableType.OUTPUT_PORT.equals(connectable.getConnectableType())) {
 connectable.getProcessGroup().stopOutputPort((Port) 
connectable);
 }
 }
 }
+
+return future;
+}
+
+@Override
+public Future activateControllerServices(final String groupId, 
final ControllerServiceState state, final Set serviceIds) {
--- End diff --

Assuming this action needs to be recorded in the Flow History, the 
`ProcessGroupAuditor` will need to account for this method invocation.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131998712
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131997934
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131978526
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/VariableRegistryDTO.java
 ---
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.web.api.dto;
+
+import java.util.Set;
+
+import javax.xml.bind.annotation.XmlType;
+
+import com.wordnik.swagger.annotations.ApiModelProperty;
+
+@XmlType(name = "variableRegistry")
+public class VariableRegistryDTO {
+private Set variables;
--- End diff --

Rather than maintaining a `Set` of `VariableDTO` with a `canUpdate` flag, 
these should probably contain a `Set` of `VariableEntity` which has the 
corresponding permissions set. This would be more consistent with existing DTOs 
for Controller Service references. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131997868
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
--- End diff --

Can we set the permissions on this entity for consistency with all of the 
other endpoints?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131993623
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
 ---
@@ -640,7 +643,128 @@ public Response scheduleComponents(
 
componentsToSchedule.entrySet().stream().collect(Collectors.toMap(Map.Entry::getKey,
 e -> getRevision(e.getValue(), e.getKey(;
 
 // update the process group
-final ScheduleComponentsEntity entity = 
serviceFacade.scheduleComponents(id, scheduledState, componentRevisions);
+final ScheduleComponentsEntity entity = 
serviceFacade.scheduleComponents(id, scheduledState, componentRevisions);
+return generateOkResponse(entity).build();
+}
+);
+}
+
+
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}/controller-services")
+@ApiOperation(value = "Enable or disable Controller Services in the 
specified Process Group.",
+response = ActivateControllerServicesEntity.class,
+authorizations = {
+@Authorization(value = "Read - /flow", type = ""),
+@Authorization(value = "Write - /{component-type}/{uuid} - For 
every service being enabled/disabled", type = "")
+})
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to 
complete the request because it was invalid. The request should not be retried 
without modification."),
+@ApiResponse(code = 401, message = "Client could not 
be authenticated."),
+@ApiResponse(code = 403, message = "Client is not 
authorized to make this request."),
+@ApiResponse(code = 404, message = "The specified 
resource could not be found."),
+@ApiResponse(code = 409, message = "The request was 
valid but NiFi was not in the appropriate state to process it. Retrying the 
same request later may be successful.")
+}
+)
+public Response activateControllerServices(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(value = "The process group id.", required = true)
+@PathParam("id") String id,
+@ApiParam(value = "The request to schedule or unschedule. If 
the comopnents in the request are not specified, all authorized components will 
be considered.", required = true)
+final ActivateControllerServicesEntity requestEntity) {
+
+// ensure the same id is being used
+if (!id.equals(requestEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process 
group id (%s) in the request body does "
++ "not equal the process group id of the requested 
resource (%s).", requestEntity.getId(), id));
+}
+
+final ControllerServiceState state;
+if (requestEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must 
be specified.");
--- End diff --

Copy/paste issue in error message... ControllerServiceState is not a 
'scheduled state'.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131996939
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -314,7 +430,7 @@ public Response updateProcessGroup(
 Authorizable authorizable = 
lookup.getProcessGroup(id).getAuthorizable();
 authorizable.authorize(authorizer, 
RequestAction.WRITE, NiFiUserUtils.getNiFiUser());
 },
-null,
+() -> 
serviceFacade.verifyUpdateProcessGroup(requestProcessGroupDTO),
--- End diff --

This is verifying that the variables can be updated but they are ignored in 
the corresponding `updateProcessGroup` call below. Since variable updates are 
not supported in this endpoint, can this check be removed?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131983295
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
 ---
@@ -3062,34 +3206,70 @@ public ProcessGroupEntity getProcessGroup(final 
String groupId) {
 return createProcessGroupEntity(processGroup);
 }
 
-private ControllerServiceEntity createControllerServiceEntity(final 
ControllerServiceNode serviceNode, final Set serviceIds) {
+private ControllerServiceEntity createControllerServiceEntity(final 
ControllerServiceNode serviceNode, final Set serviceIds, final NiFiUser 
user) {
 final ControllerServiceDTO dto = 
dtoFactory.createControllerServiceDto(serviceNode);
 
 final ControllerServiceReference ref = serviceNode.getReferences();
 final ControllerServiceReferencingComponentsEntity 
referencingComponentsEntity = 
createControllerServiceReferencingComponentsEntity(ref, serviceIds);
 
dto.setReferencingComponents(referencingComponentsEntity.getControllerServiceReferencingComponents());
 
 final RevisionDTO revision = 
dtoFactory.createRevisionDTO(revisionManager.getRevision(serviceNode.getIdentifier()));
-final PermissionsDTO permissions = 
dtoFactory.createPermissionsDto(serviceNode);
+final PermissionsDTO permissions = 
dtoFactory.createPermissionsDto(serviceNode, user);
 final List bulletins = 
dtoFactory.createBulletinDtos(bulletinRepository.findBulletinsForSource(serviceNode.getIdentifier()));
 final List bulletinEntities = 
bulletins.stream().map(bulletin -> entityFactory.createBulletinEntity(bulletin, 
permissions.getCanRead())).collect(Collectors.toList());
 return entityFactory.createControllerServiceEntity(dto, revision, 
permissions, bulletinEntities);
 }
 
 @Override
-public Set getControllerServices(final String 
groupId) {
-final Set serviceNodes = 
controllerServiceDAO.getControllerServices(groupId);
+public VariableRegistryEntity getVariableRegistry(final String 
groupId) {
+final ProcessGroup processGroup = 
processGroupDAO.getProcessGroup(groupId);
+if (processGroup == null) {
+throw new ResourceNotFoundException("Could not find group with 
ID " + groupId);
+}
+
+return createVariableRegistryEntity(processGroup);
+}
+
+private VariableRegistryEntity createVariableRegistryEntity(final 
ProcessGroup processGroup) {
+final VariableRegistryDTO registryDto = 
dtoFactory.createVariableRegistryDto(processGroup);
+final RevisionDTO revision = 
dtoFactory.createRevisionDTO(revisionManager.getRevision(processGroup.getIdentifier()));
+final PermissionsDTO permissions = 
dtoFactory.createPermissionsDto(processGroup);
+return entityFactory.createVariableRegistryEntity(registryDto, 
revision, permissions);
+}
+
+@Override
+public VariableRegistryEntity populateAffectedComponents(final 
VariableRegistryDTO variableRegistryDto) {
--- End diff --

`NiFiServiceFacadeLock` handles read/write locking based on the method 
name. Assuming this method needs that thread safety, the name of this method 
needs to be accounted for. Either by changing it here or adding another advice 
to intercept this method call.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131983144
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
 ---
@@ -821,6 +925,46 @@ public ScheduleComponentsEntity 
scheduleComponents(final String processGroupId,
 }
 
 @Override
+public ActivateControllerServicesEntity 
activateControllerServices(final String processGroupId, final 
ControllerServiceState state, final Map serviceRevisions) {
--- End diff --

`NiFiServiceFacadeLock` handles read/write locking based on the method 
name. Assuming this method needs that thread safety, the name of this method 
needs to be accounted for. Either by changing it here or adding another advice 
to intercept this method call.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131985592
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardProcessGroupDAO.java
 ---
@@ -145,6 +202,21 @@ public ProcessGroup updateProcessGroup(ProcessGroupDTO 
processGroupDTO) {
 }
 
 @Override
+public ProcessGroup updateVariableRegistry(final VariableRegistryDTO 
variableRegistry) {
--- End diff --

Assuming this action needs to be recorded in the Flow History, the 
`ProcessGroupAuditor` will need to account for this method invocation.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131979395
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessGroupDTO.java
 ---
@@ -200,4 +204,13 @@ public void setInactiveRemotePortCount(Integer 
inactiveRemotePortCount) {
 this.inactiveRemotePortCount = inactiveRemotePortCount;
 }
 
+
+@ApiModelProperty("The variables that are configured for the Process 
Group")
+public Map getVariables() {
+return variables;
+}
+
+public void setVariables(final Map variables) {
--- End diff --

These should be set when populating the DTO in 
`DtoFactory.createConciseProcessGroupDto(...)`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132005935
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131983043
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
 ---
@@ -774,6 +791,80 @@ public RemoteProcessGroupPortEntity 
updateRemoteProcessGroupOutputPort(
 }
 
 @Override
+public Set 
identifyComponentsAffectedByVariableRegistryUpdate(final VariableRegistryDTO 
variableRegistryDto) {
--- End diff --

`NiFiServiceFacadeLock` handles read/write locking based on the method 
name. Assuming this method needs that thread safety, the name of this method 
needs to be accounted for. Either by changing it here or adding another advice 
to intercept this method call. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131998273
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131979744
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -140,6 +185,22 @@
 private TemplateResource templateResource;
 private ControllerServiceResource controllerServiceResource;
 
+private final DtoFactory dtoFactory = new DtoFactory();
--- End diff --

If using the `DtoFactory` here, it can be injected in the Spring context.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131999511
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131994112
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
 ---
@@ -640,7 +643,128 @@ public Response scheduleComponents(
 
componentsToSchedule.entrySet().stream().collect(Collectors.toMap(Map.Entry::getKey,
 e -> getRevision(e.getValue(), e.getKey(;
 
 // update the process group
-final ScheduleComponentsEntity entity = 
serviceFacade.scheduleComponents(id, scheduledState, componentRevisions);
+final ScheduleComponentsEntity entity = 
serviceFacade.scheduleComponents(id, scheduledState, componentRevisions);
+return generateOkResponse(entity).build();
+}
+);
+}
+
+
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}/controller-services")
+@ApiOperation(value = "Enable or disable Controller Services in the 
specified Process Group.",
+response = ActivateControllerServicesEntity.class,
+authorizations = {
+@Authorization(value = "Read - /flow", type = ""),
+@Authorization(value = "Write - /{component-type}/{uuid} - For 
every service being enabled/disabled", type = "")
+})
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to 
complete the request because it was invalid. The request should not be retried 
without modification."),
+@ApiResponse(code = 401, message = "Client could not 
be authenticated."),
+@ApiResponse(code = 403, message = "Client is not 
authorized to make this request."),
+@ApiResponse(code = 404, message = "The specified 
resource could not be found."),
+@ApiResponse(code = 409, message = "The request was 
valid but NiFi was not in the appropriate state to process it. Retrying the 
same request later may be successful.")
+}
+)
+public Response activateControllerServices(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(value = "The process group id.", required = true)
+@PathParam("id") String id,
+@ApiParam(value = "The request to schedule or unschedule. If 
the comopnents in the request are not specified, all authorized components will 
be considered.", required = true)
+final ActivateControllerServicesEntity requestEntity) {
+
+// ensure the same id is being used
+if (!id.equals(requestEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process 
group id (%s) in the request body does "
++ "not equal the process group id of the requested 
resource (%s).", requestEntity.getId(), id));
+}
+
+final ControllerServiceState state;
+if (requestEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must 
be specified.");
+} else {
+try {
+state = 
ControllerServiceState.valueOf(requestEntity.getState());
+} catch (final IllegalArgumentException iae) {
+throw new IllegalArgumentException(String.format("The 
scheduled must be one of [%s].",
--- End diff --

Looks like another copy/paste in this error message. Also, appears to be 
missing 'state' in where the message was copied from.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-08 Thread mcgilman 
Github user mcgilman commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r131983689
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
 ---
@@ -821,6 +925,46 @@ public ScheduleComponentsEntity 
scheduleComponents(final String processGroupId,
 }
 
 @Override
+public ActivateControllerServicesEntity 
activateControllerServices(final String processGroupId, final 
ControllerServiceState state, final Map serviceRevisions) {
+
+final NiFiUser user = NiFiUserUtils.getNiFiUser();
+return activateControllerServices(user, processGroupId, state, 
serviceRevisions);
+}
+
+@Override
+public ActivateControllerServicesEntity 
activateControllerServices(final NiFiUser user, final String processGroupId, 
final ControllerServiceState state,
--- End diff --

`NiFiServiceFacadeLock` handles read/write locking based on the method 
name. Assuming this method needs that thread safety, the name of this method 
needs to be accounted for. Either by changing it here or adding another advice 
to intercept this method call.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

2017-08-03 Thread markap14 
GitHub user markap14 opened a pull request:

https://github.com/apache/nifi/pull/2051

NIFI-4224: Initial implementation of Process Group level Variable Reg…

…istry

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [ ] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markap14/nifi NIFI-4224

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2051.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2051


commit 3c43dedb2038a3c3e674c0be876c43010d672938
Author: Mark Payne 
Date:   2017-07-25T17:46:10Z

NIFI-4224: Initial implementation of Process Group level Variable Registry




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---