[GitHub] nifi issue #1625: NIFI-3650 - Adjust travis to forcefuly remove $HOME/.m2/re...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1625 @apiri I think there might be two tasks here to make sure everything is copacetic: (1) remove that part of the tree so we don't bother caching it/downloading it period and (2) removing it before the build just to make sure it's not there (will need that the first time this runs anyway if we don't invalidate the cache). So I think it makes sense to add this but also keep the other. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1625: NIFI-3650 - Adjust travis to forcefuly remove $HOME/.m2/re...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1625 @apiri Oh, I see now. You're saying move the one in `script` to `before_cache`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1312 @kedarchitale +1 will merge in the morning. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1663: NIFI-3692 Use hbc-core instead of hbc-twitter4j in ...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1663 NIFI-3692 Use hbc-core instead of hbc-twitter4j in social-media bundle to resolve org.json cat-x Note: We can't remove Twitter4j from nifi-assembly/NOTICE until [NIFI-3018](https://issues.apache.org/jira/browse/NIFI-3018) is complete. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-3692 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1663.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1663 commit 2c3a1215ccee9edd213a55dba0d78a9407b36413 Author: Joey Frazee Date: 2017-04-11T15:19:44Z NIFI-3692 Use hbc-core instead of hbc-twitter4j in social-media bundle to resolve org.json cat-x --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1015: NIFI-2767 Periodically reload properties from file-based v...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1015 @trixpan Just rebased it again. Made some minor changes: using MD5SumMonitor with the watcher now because the last modified monitor wasn't reliably testable, setting the check schedule to 0 disables it, and cleaned up the tests. In the JIRA the question of whether this was safe was brought up. From the standpoint of will reloading the map in the FileBasedVariableRegistry break anything core, I think the answer is no (it's only used in EL value lookups), so, assuming I didn't make anything too buggy, it's safe in that respect. From the standpoint of will it be possible to build flows that don't behave like the author was expecting if they expected stuff to be static, the answer is probably yes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1696: NIFI-1655 - Add .gitattributes to specifically define
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1696 @trixpan You're right about the HL7 and it's actually ill-formed if you don't use CR, crazy. For best results, though, we should just move the HL7 content into a String in the test class. Renaming/dealing in git is okayish, but I think sets us up to have pain here and there. I opened [NIFI-3771](https://issues.apache.org/jira/browse/NIFI-3771) to address. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1731: NIFI-3771 Move HL7 test docs into class members to ...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1731 NIFI-3771 Move HL7 test docs into class members to avoid git rewriting CRs See also [NIFI-1655](https://issues.apache.org/jira/browse/NIFI-1655) and #1696 You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-3771 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1731.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1731 commit e5a89b1145f9c5a47bc53c0b6626ced2dad85069 Author: Joey Frazee Date: 2017-05-02T15:48:10Z NIFI-3771 Move HL7 test docs into class members to avoid git rewriting CRs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1696: NIFI-1655 - Add .gitattributes to specifically define
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1696 @trixpan Do you know which files are problematic? I'm assuming the *.txt; i.e., the content that's being encrypted. I'd figure it wouldn't matter for the *.asc and *.pgp. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1763: NIFI-3815 Fix build RPM using mvn install -Prpm,gen...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1763 NIFI-3815 Fix build RPM using mvn install -Prpm,generateArchives Note that GPG signing may not work with certain combinations of the Maven plugin, GPG and RPM build tools [1]. 1. https://github.com/mojohaus/rpm-maven-plugin/issues/44 You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-3815 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1763.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1763 commit 7cd6dcc2e75fa7075092f81063119fced9c0b314 Author: Joey Frazee Date: 2017-05-05T18:10:14Z NIFI-3815 Fix build RPM using mvn install -Prpm,generateArchives --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1807: NIFI-3910: Fixed issue where CSVRecordReader throws...
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1807#discussion_r116796084
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/CSVRecordReader.java
---
@@ -68,17 +68,19 @@ public Record nextRecord() throws IOException,
MalformedRecordException {
final Map rowValues = new
HashMap<>(schema.getFieldCount());
for (final RecordField recordField : schema.getFields()) {
-String rawValue =
csvRecord.get(recordField.getFieldName());
-if (rawValue == null) {
+String rawValue = null;
+final String fieldName = recordField.getFieldName();
+if (csvRecord.isSet(fieldName)) {
+rawValue = csvRecord.get(fieldName);
+} else {
for (final String alias : recordField.getAliases()) {
-rawValue = csvRecord.get(alias);
-if (rawValue != null) {
+if (csvRecord.isSet(fieldName)) {
--- End diff --
@markap14 Shouldn't you be checking isSet(alias)? Since this is in the
else, presumably isSet(fieldName) is still false?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1807: NIFI-3910: Fixed issue where CSVRecordReader throws an Exc...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1807 I'm ok on a +1 once that alias thing gets fixed, unless, it's correct as is. Tested with a simple flow before/after this patch and the exception is gone now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1809: NIFI-3912: Fixed NPE that would result in validation failu...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1809 With this FreeFormTextRecordSetWriter enables correctly now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1830: NIFI-3404 Add lookup processor for enrichments/join...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1830 NIFI-3404 Add lookup processor for enrichments/joins to reference data Note: This is overlaid on [NIFI-3339](https://issues.apache.org/jira/browse/NIFI-3339)/#1450. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-3404 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1830.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1830 commit 449f33ca5379b22f8dc4b7ba0042481c46129045 Author: Joey Frazee Date: 2017-05-19T16:45:14Z NIFI-3404 Added properties and csv file lookup services commit f2cda5c47c19b1d95a8813c2c79329f49537da1a Author: Joey Frazee Date: 2017-05-19T17:49:24Z NIFI-3404 Added LookupAttribute processor, L&N fixes commit 64e03e344432665b7b4184405335f62221c3e93a Author: Toivo Adams Date: 2017-01-29T11:40:37Z NIFI-3339 Add getDataSource() to DBCPService This closes #1450 Signed-off-by: Joey Frazee commit c3b7e2e76eb38500c02e42676cfe1bcaa0b2291d Author: Joey Frazee Date: 2017-05-19T18:32:41Z NIFI-3404 Added database lookup service --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1833: NIFI-3946: Update LookupService to take a Map inste...
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1833#discussion_r117601695
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/main/java/org/apache/nifi/lookup/SimpleKeyValueLookupService.java
---
@@ -52,7 +57,21 @@ public void cacheConfiguredValues(final
ConfigurationContext context) {
}
@Override
-public Optional lookup(final String key) {
+public Optional lookup(final Map coordinates) {
+if (coordinates == null) {
+return null;
--- End diff --
Should these nulls be Optional.empty()?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1833: NIFI-3946: Update LookupService to take a Map inste...
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1833#discussion_r117601703
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/main/java/org/apache/nifi/lookup/maxmind/IPLookupService.java
---
@@ -146,17 +153,27 @@ public void closeReader() throws IOException {
}
@Override
-public Optional lookup(final String key) throws
LookupFailureException {
-if (key == null) {
+public Set getRequiredKeys() {
+return REQUIRED_KEYS;
+}
+
+@Override
+public Optional lookup(final Map coordinates)
throws LookupFailureException {
+if (coordinates == null) {
return Optional.empty();
}
+final String ipAddress = coordinates.get(IP_KEY);
+if (ipAddress == null) {
+return null;
--- End diff --
Optional.empty() here too?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1830: NIFI-3404 Add lookup processor for enrichments/joins to re...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1830 @ijokarumawak FYI, I think we need to make sure we see what happens to [NIFI-3946](https://issues.apache.org/jira/browse/NIFI-3946)/#1833 before we merge this. By all means start having a look though. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1830: NIFI-3404 Add lookup processor for enrichments/join...
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1830#discussion_r117809657
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-lookup-service-api/src/main/java/org/apache/nifi/lookup/LookupService.java
---
@@ -17,25 +17,47 @@
package org.apache.nifi.lookup;
+import java.util.Map;
import java.util.Optional;
+import java.util.Set;
import org.apache.nifi.controller.ControllerService;
public interface LookupService extends ControllerService {
/**
- * Looks up a value that corresponds to the given key
+ * Looks up a value that corresponds to the given map of information,
referred to as lookup coordinates
*
- * @param key the key to lookup
- * @return a value that corresponds to the given key
+ * @param coordinates a Map of key/value pairs that indicate the
information that should be looked up
+ * @return a value that corresponds to the given coordinates
*
- * @throws LookupFailureException if unable to lookup a value for the
given key
+ * @throws LookupFailureException if unable to lookup a value for the
given coordinates
*/
-Optional lookup(String key) throws LookupFailureException;
+Optional lookup(Map coordinates) throws
LookupFailureException;
/**
- * @return the Class that represents the type of value that will be
returned by {@link #lookup(String)}
+ * Returns the entire lookup table as a {@link Map}
+ *
+ * @return the lookup table stored in the backing service
+ * @throws LookupFailureException if the backing service is
unavailable or
+ * the table cannot be loaded
+ */
+default Map asMap() throws LookupFailureException {
--- End diff --
I think/know there's a use case for this but I get the worry, so I'll drop
it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1830: NIFI-3404 Add lookup processor for enrichments/join...
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1830#discussion_r117809682
--- Diff:
nifi-nar-bundles/nifi-lookup-bundle/nifi-lookup-processors/src/main/java/org/apache/nifi/processors/lookup/LookupAttribute.java
---
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.lookup;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+
+import org.apache.commons.lang3.StringUtils;
+
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.expression.AttributeExpression;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.lookup.LookupFailureException;
+import org.apache.nifi.lookup.LookupService;
+import org.apache.nifi.lookup.StringLookupService;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"lookup", "cache", "enrich", "join", "mutable", "attributes",
"Attribute Expression Language"})
+@CapabilityDescription("Lookup attributes from a lookup service")
+@DynamicProperty(name = "The name of the attribute to add to the FlowFile",
+value = "The name of the key or property to retrieve from the lookup
service",
+supportsExpressionLanguage = true,
+description = "Adds a FlowFile attribute specified by the dynamic
property's key with the value found in the lookup service using the the dynamic
property's value")
+@WritesAttribute(attribute = "See additional details", description = "This
processor may write zero or more attributes as described in additional details")
+public class LookupAttribute extends AbstractProcessor {
+
+public static final PropertyDescriptor LOOKUP_SERVICE =
+new PropertyDescriptor.Builder()
+.name("lookup-service")
+.displayName("Lookup Service")
+.description("The lookup service to use for attribute lookups")
+.identifiesControllerService(StringLookupService.class)
+.required(true)
+.build();
+
+public static final PropertyDescriptor INCLUDE_EMPTY_VALUES =
+new PropertyDescriptor.Builder()
+.name("include-empty-values")
+.displayName("Include Empty Values")
+.description("Include null or blank values for keys that are
null or blank")
+.addValidator(St
[GitHub] nifi pull request #1830: NIFI-3404 Add lookup processor for enrichments/join...
Github user jfrazee commented on a diff in the pull request: https://github.com/apache/nifi/pull/1830#discussion_r117810505 --- Diff: nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/main/java/org/apache/nifi/lookup/SimpleKeyValueLookupService.java --- @@ -41,7 +48,9 @@ protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String .name(propertyDescriptorName) .required(false) .dynamic(true) -.addValidator(Validator.VALID) + .addValidator(StandardValidators.createAttributeExpressionLanguageValidator(AttributeExpression.ResultType.STRING, true)) + .addValidator(StandardValidators.ATTRIBUTE_KEY_PROPERTY_NAME_VALIDATOR) +.expressionLanguageSupported(true) --- End diff -- You're right, I was trying to merge changes in from a VolatileLookupService I wrote elsewhere and didn't get that updated. The use for this is a little marginal so I'll remove it and we can see if the need comes back. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1794: NIFI-1709 - Introduce logic to probe Linux version using /...
Github user jfrazee commented on the issue:
https://github.com/apache/nifi/pull/1794
@trixpan @apiri Few things. It seems like we should be using lsb_release to
grab the OS as well as chkconfig to handle creating run level symlinks instead
of on on our own, at least for most of the OSes. And, while we install an
init.d script and systemd can run those, we actually don't create a systemd
unit today, so for many of the latest releases service and systemctl won't
actually start nifi.
So I guess what I'm saying is that I think there might be 3 issues here: OS
detection improvement, change how we set run levels, and add systemd unit and
upstart scripts for OSes that aren't really doing sysv init anymore.
FWIW I've been using something like this for the systemd unit:
```
[Unit]
Description=Apache NiFi
After=network.target
[Service]
Type=forking
User={{ nifi_run_as }}
Group={{ nifi_run_as }}
ExecStart={{ nifi_install_dir }}/bin/nifi.sh start
ExecStop={{ nifi_install_dir }}/bin/nifi.sh stop
Restart=on-failure
RestartSec=60s
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1830: NIFI-3404 Add lookup processor for enrichments/join...
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1830#discussion_r118341314
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/LookupAttribute.java
---
@@ -0,0 +1,289 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+
+import org.apache.commons.lang3.StringUtils;
+
+import org.apache.nifi.annotation.behavior.DynamicProperty;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.expression.AttributeExpression;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.lookup.LookupFailureException;
+import org.apache.nifi.lookup.LookupService;
+import org.apache.nifi.lookup.StringLookupService;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"lookup", "cache", "enrich", "join", "mutable", "attributes",
"Attribute Expression Language"})
--- End diff --
Holdover from the interfaces this was originally written against. There was
a LookupService and a MutableLookupService. Will remove.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1830: NIFI-3404 Add lookup processor for enrichments/joins to re...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1830 @ijokarumawak The changes from [NIFI-3339](https://issues.apache.org/jira/browse/NIFI-3339)/#1450 have been removed from this PR along with the DatabaseLookupService. Will re-submit that stuff in another PR so we can move ahead with this and work out the details separately. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1830: NIFI-3404 Add lookup processor for enrichments/join...
Github user jfrazee commented on a diff in the pull request: https://github.com/apache/nifi/pull/1830#discussion_r118408976 --- Diff: nifi-nar-bundles/nifi-standard-services/nifi-lookup-services-bundle/nifi-lookup-services/src/main/java/org/apache/nifi/lookup/SimpleKeyValueLookupService.java --- @@ -47,6 +47,7 @@ protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String .required(false) .dynamic(true) .addValidator(Validator.VALID) +.expressionLanguageSupported(true) --- End diff -- I meant to remove the expressionLanguageSupported(true). I don't think I had an especially good reason to have added the EL support to the dynamic property for the CS in the first place. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1794: NIFI-1709 - Introduce logic to probe Linux version using /...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1794 @trixpan Fair enough. Seems clear the lsb_release suggestion was bad. BTW, I tested your changes on opensuse:latest (leap) and it's working as expected. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1794: NIFI-1709 - Introduce logic to probe Linux version using /...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1794 @trixpan Yep, it's working. Is there any use in using the ID_LIKE variable instead of ID? If so we could change that up quick, else I think this is good to go. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1858: NIFI-3978 Increase threadpool size for S2S HTTP tes...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1858 NIFI-3978 Increase threadpool size for S2S HTTP tests Increases threadpool limits for S2S HTTP tests so they'll pass on many core machines, as discussed in [NIFI-3978](https://issues.apache.org/jira/browse/NIFI-3978). You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-3978 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1858.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1858 commit bea2ddaf10f88832eadefbbd036b25bfabe47c72 Author: Joey Frazee Date: 2017-05-25T15:32:06Z NIFI-3978 Increase threadpool size for S2S HTTP tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1830: NIFI-3404 Add lookup processor for enrichments/join...
Github user jfrazee closed the pull request at: https://github.com/apache/nifi/pull/1830 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1830: NIFI-3404 Add lookup processor for enrichments/joins to re...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1830 Superseded by #1856 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1856: NIFI-3404: LookupAttribute, SimpleCsvFileLookupService, Pr...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1856 This was a +1 btw, obviously. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1865: NIFI-3836 Added check for root user in test that depends o...
Github user jfrazee commented on the issue:
https://github.com/apache/nifi/pull/1865
@alopresto I was expecting to see System.getProperty("user.name") used
instead of the exec. But I'm a little naive about what the drawback could be;
maybe because you could override it and then user.name isn't actually the user.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1865: NIFI-3836 Added check for root user in test that de...
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1865#discussion_r118769414
--- Diff:
nifi-commons/nifi-data-provenance-utils/src/test/groovy/org/apache/nifi/provenance/CryptoUtilsTest.groovy
---
@@ -88,6 +88,11 @@ class CryptoUtilsTest {
Cipher.getMaxAllowedKeyLength("AES") > 128
}
+private static boolean isRootUser() {
+Process p = Runtime.getRuntime().exec("id -u")
+p.exitValue() == 0
--- End diff --
Isn't this just going to be true whenever `id -u` succeeds?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1865: NIFI-3836 Added check for root user in test that depends o...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1865 Reviewing... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1865: NIFI-3836 Added check for root user in test that depends o...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1865 @alopresto Welp, TestGetFile has similar problems. This one LGTM so question is whether we want to merge this one or expand the scope; thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1794: NIFI-1709 - Introduce logic to probe Linux version using /...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1794 @trixpan LGTM +1 and I tested on SuSE and not SuSE. You can go ahead and merge it or I can in my afternoon/your evening. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1865: NIFI-3836 Added check for root user in test that depends o...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1865 @alopresto +1 will open another JIRA for covering the rest of the problems. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #609: NIFI-2142 Cache compiled XSLT in TransformXml
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/609 @JPercivall Oh, right. I think these are corrected now but just noticed the image bundle has a bunch of license violations (even after rebasing). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #565: NIFI-2090 Add support for HL7 segment names and comp...
Github user jfrazee closed the pull request at: https://github.com/apache/nifi/pull/565 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #565: NIFI-2090 Add support for HL7 segment names and components ...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/565 Closing to resubmit against master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #669: NIFI-2090 Add support for HL7 segment names and comp...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/669 NIFI-2090 Add support for HL7 segment names and components in ExtractHL7Attributes Note that this doesn't apply the "friendly" names to for composite field types and those are still produced with numeric indexes (e.g., PID.PatientIDInternalID.CX.1). While that's super useful, the only way to get at that with the underlying HAPI library is via reflection, so it will require a separate commit with more extensive testing. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-2090 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/669.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #669 commit 0dc6a278a851334db734363e47165b49df5e9410 Author: Joey Frazee Date: 2016-06-23T04:44:40Z Added options for segment names, parse fields in ExtractHL7Attributes commit 717bd6c09234d609e004de48015d043dbf6ef977 Author: Joey Frazee Date: 2016-06-23T05:04:25Z Fix mislabeled parse-segment-fields property displayName --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #609: NIFI-2142 Cache compiled XSLT in TransformXml
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/609#discussion_r71950331
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformXml.java
---
@@ -76,25 +84,59 @@
.name("XSLT file name")
.description("Provides the name (including full path) of the
XSLT file to apply to the flowfile XML content.")
.required(true)
+.expressionLanguageSupported(true)
.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
.build();
+public static final PropertyDescriptor INDENT_OUTPUT = new
PropertyDescriptor.Builder()
+.name("indent-output")
+.displayName("Indent")
+.description("Whether or not to indent the output.")
+.required(true)
+.defaultValue("true")
+.allowableValues("true", "false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor CACHE_SIZE = new
PropertyDescriptor.Builder()
+.name("cache-size")
+.displayName("Cache size")
+.description("Maximum size of the stylesheet cache.")
+.required(true)
+.defaultValue("100")
+
.addValidator(StandardValidators.NON_NEGATIVE_INTEGER_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor CACHE_DURATION = new
PropertyDescriptor.Builder()
+.name("cache-duration")
+.displayName("Cache duration")
+.description("How long to keep stylesheets in the cache.")
--- End diff --
@JPercivall It can be either. Do you have thoughts on which is preferable.
If kept as is, how does "How long to keep stylesheets in the cache after last
time of access" sound?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #609: NIFI-2142 Cache compiled XSLT in TransformXml
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/609#discussion_r71950397
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/TransformXml.java
---
@@ -76,25 +84,59 @@
.name("XSLT file name")
.description("Provides the name (including full path) of the
XSLT file to apply to the flowfile XML content.")
.required(true)
+.expressionLanguageSupported(true)
.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
.build();
+public static final PropertyDescriptor INDENT_OUTPUT = new
PropertyDescriptor.Builder()
+.name("indent-output")
+.displayName("Indent")
+.description("Whether or not to indent the output.")
+.required(true)
+.defaultValue("true")
+.allowableValues("true", "false")
+.addValidator(StandardValidators.BOOLEAN_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor CACHE_SIZE = new
PropertyDescriptor.Builder()
+.name("cache-size")
+.displayName("Cache size")
+.description("Maximum size of the stylesheet cache.")
+.required(true)
+.defaultValue("100")
+
.addValidator(StandardValidators.NON_NEGATIVE_INTEGER_VALIDATOR)
--- End diff --
@JPercivall Yes, that's correct. So "Maximum number of stylesheets to store
in the cache"?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #609: NIFI-2142 Cache compiled XSLT in TransformXml
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/609 1. @JPercivall I addressed all your comments above and added a template to https://gist.github.com/jfrazee/a501336826aa77d0daa74e8e0aa3afd0. 2. There is one change in behavior now -- a cache size of 0 disables the cache whereas it was just allowing for an arbitrarily large cache. 3. There's a screenshot of the example data flow added to https://issues.apache.org/jira/browse/NIFI-2142 which illustrates the difference between running with and without caching. It's not 100% apples to apples with the older TransformXml, but behaves the same way. The queues back up and there's a stark contrast in Tasks/Time. 4. I rebased on the latest master, but contrib-check is failing because of the commenting style on AbstractSNMPProcessor. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #609: NIFI-2142 Cache compiled XSLT in TransformXml
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/609 @JPercivall Addressed your latest comments and added a test with the cache disabled since the logic varies depending on that value now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #854: NIFI-2564 Add support for HL7 composite component na...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/854 NIFI-2564 Add support for HL7 composite component names Wanted to start getting some feedback on this. ExtractHL7Attributes can currently parse composite components, e.g., output PID.PatientName.XPN.2, which corresponds to the patient's given name, but it'd be more helpful if the attribute was something like PID.PatientName.GivenName or PID.PatientName.XPN.GivenName. Unfortunately, the underlying library doesn't provide a common way or interfaces around the composite types that make it possible to get at that, so this change gets at them with reflection. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-2564 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/854.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #854 commit 0a381e91e76976204e3fdd26e81c930575a4278b Author: Joey Frazee Date: 2016-07-18T21:31:41Z Propagate Use Segment Names property down to HL7 segment components --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1406: NIFI-3302 QueryDatabaseTable incorrectly says it ca...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1406 NIFI-3302 QueryDatabaseTable incorrectly says it can be triggered by an incoming FlowFile - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-3302 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1406.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1406 commit b40cbf1ed1298c02ecbb9fb138e18298c281e37d Author: Joey Frazee Date: 2017-01-09T01:44:08Z NIFI-3302 Remove 'accepts incoming FlowFile' doc from QDT --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1422: NIFI-3015 - Corrects issue were RPM profile run.as propert...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1422 @trixpan Starting review of this right now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1422: NIFI-3015 - Corrects issue were RPM profile run.as ...
Github user jfrazee commented on a diff in the pull request: https://github.com/apache/nifi/pull/1422#discussion_r96305976 --- Diff: nifi-assembly/pom.xml --- @@ -19,6 +19,10 @@ language governing permissions and limitations under the License. --> nifi-assembly pom This is the assembly Apache NiFi + + + --- End diff -- @trixpan Including this at the top level means that with the rpm profile, the bootstrap.conf included in the .tar.gz and .zip packages has run.as=nifi. Is this the right behavior? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1422: NIFI-3015 - Corrects issue were RPM profile run.as ...
Github user jfrazee commented on a diff in the pull request: https://github.com/apache/nifi/pull/1422#discussion_r96313139 --- Diff: nifi-assembly/pom.xml --- @@ -19,6 +19,10 @@ language governing permissions and limitations under the License. --> nifi-assembly pom This is the assembly Apache NiFi + + + --- End diff -- @trixpan I guess what I'm saying is that with this change I think -Prpm makes a good rpm but not a good .tar.gz or .zip since the user will now need to edit bootstrap.conf if they're using the archive package; install.sh doesn't create the nifi user, right? I think somehow or other it needs to use the defaultUsername that the maven-rpm-plugin is using but I'm not sure how without digging further. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1422: NIFI-3015 - Corrects issue were RPM profile run.as ...
Github user jfrazee commented on a diff in the pull request: https://github.com/apache/nifi/pull/1422#discussion_r96983531 --- Diff: nifi-assembly/pom.xml --- @@ -19,6 +19,10 @@ language governing permissions and limitations under the License. --> nifi-assembly pom This is the assembly Apache NiFi + + + --- End diff -- @trixpan So from my point of view running the rpm profile should be additive and not change the .tar.gz or .zip. I think we can use installScriptlet or postinstallScriptlet to do a sed replace of the run.as= and not have to fuss with modifying bootstrap.conf outside of the rpm. Thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1312 @kedarchitale I'm going to try picking up the review on this so we can get this done. Thanks for already making all the changes Joe suggested! I have two questions now. First, do you think there's any sense in moving this over into the existing HL7 bundle? I don't have strong opinions but I think keeping the HL7 related stuff together would be good. Would love to know your perspective. Second, I can't verify the source + licensing of the Transition_of_Care_Referral_Summary.xml example file that's used. Based on your docs I know you got it from chb/sample_ccdas (thanks for making that clear too) but it's not obvious that we can say it's ASLv2. Tagging @jmandel who I think made the contrib to that repo in case he can help clear it up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1422: NIFI-3015 - Corrects issue were RPM profile run.as propert...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1422 LGTM +1 Verified builds on Amazon Linux w/ and w/o -Prpm and checked rpm vs. tar.gz and .zip installs to verify correct bootstrap.conf. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1312 @kedarchitale Thanks for the explanation. I just wanted to make sure we discussed the packaging. The XML file is potentially an issue though. Even if it's an open license, without knowing what the contributor intended the particular license to be, we don't know if it's actually compatible with ASLv2. Since this will end up in the source release we have to be 100% sure about this. I want to get this wrapped up this week so I'm going to try to email @jmandel and @gjewell and see if they can clarify. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1312 @kedarchitale Fantastic. Will start digging into the changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request: https://github.com/apache/nifi/pull/1312#discussion_r102478227 --- Diff: nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/pom.xml --- @@ -0,0 +1,83 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-ccda-bundle +1.2.0-SNAPSHOT + + +nifi-ccda-processors +jar + + + +org.apache.nifi +nifi-api + + +org.apache.nifi +nifi-processor-utils + + --- End diff -- These need to be de-indented. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102478735
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/resources/docs/org.apache.nifi.processors.ccda.ExtractCCDAAttributes/additionalDetails.html
---
@@ -0,0 +1,108 @@
+
+
+
+
+
+ExtractCCDAAttributes
+
+
+
+
+
+Apache NiFi CCDA Processor
+
+CCDA Processor Bundle provides parser for Consolidated-CDA
documents
+This processor provides JSON in FlowFile content. Pretty
Printing can be set as a property while configuring the processor.
+Individual attributes are set as FlowFile attributes. The
attributes are named as <Parent> <dot> <Key>.
+If the Parent is repeating, the naming will be <Parent>
<underscore> <Parent Index> <dot> <Key>.
+
+Example JSON Output
+
+
+"observation": [
+ {
+"id": [
+ {
+"extension": "121387536",
+"root": "2.16.840.1.113883.1.13.9.999239"
+ }
+],
+"values": {
+ "code": "194828000",
+ "codeSystem": "2.16.840.1.113883.6.96",
+ "codeSystemName": "SNOMED CT",
+ "displayName": "Angina (disorder)",
+ "translations": [
+{
+ "code": "413.9",
+ "codeSystem": "2.16.840.1.113883.6.103",
+ "codeSystemName": "ICD-9CM (diagnosis codes)",
+ "displayName": "Other and unspecified angina pectoris"
+}
+ ]
+},
+"statusCode": {
+ "code": "completed"
+},
+"effectiveTime": {
+ "low": "20130711"
+}
+ }
+]
+
+
+Example Attribute Output
+
+
+problemSection.act_04.observation.effectiveTime.low=20130711
+problemSection.act_04.observation.id.extension=121387536
+
problemSection.act_04.observation.id.root=2.16.840.1.113883.1.13.9.999239
+problemSection.act_04.observation.statusCode.code=completed
+problemSection.act_04.observation.values.code=194828000
+
problemSection.act_04.observation.values.codeSystem=2.16.840.1.113883.6.96
+problemSection.act_04.observation.values.codeSystemName=SNOMED CT
+problemSection.act_04.observation.values.displayName=Angina (disorder)
+problemSection.act_04.observation.values.translations.code=413.9
+
problemSection.act_04.observation.values.translations.codeSystem=2.16.840.1.113883.6.103
+
problemSection.act_04.observation.values.translations.codeSystemName=ICD-9CM
(diagnosis codes)
+
problemSection.act_04.observation.values.translations.displayName=Other and
unspecified angina pectoris
+problemSection.act_04.statusCode.code=active
+
+
+Example Parser Mapping
+
+This processor is driven by a mapping file which specifies the
element relationships. For example
+
+
org.openhealthtools.mdht.uml.cda.consol.impl.ProblemObservationImpl=id#element.ids\
+@values#element.values[0]\
+@statusCode#element.statusCode\
+@effectiveTime#element.effectiveTime\
+@negation#element.negationInd\
+@problemStatus#element.problemStatus
+
org.openhealthtools.mdht.uml.cda.consol.impl.ProblemStatusImpl=id#element.id\
+@code#element.code\
+@values#element.values[0]\
+@statusCode#element.statusCode
+
+
+References
+
+These mappings are defined as per the implementation guide -http://www.hl7.org/documentcenter/public/standards/dstu/CDAR2_IG_IHE_CONSOL_DSTU_R1dot1_2012JUL.zip";>HL7
Implementation Guide for CDA® Release 2
+
+CCDA sample file used in this bundle for testing is from https://github.com/chb/sample_ccdas/blob/master/Cerner%20Samples/Transition_of_Care_Referral_Summary.xml";>Cerner
Sample - Transition of Care Referral Summary
--- End diff --
Since the Cerner test file isn't included in the testing anymore, need to
remove this from the additionalDetails.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102486355
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "JSON", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides JSON as FlowFile content "
++ "and individual attributes as FlowFile attributes. The
attributes are named as . "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final String APPLICATION_JSON = "application/json";
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private List timingStats = new ArrayList(); // stores
timing statistics
--- End diff --
Instead of timingStats use
[StopWatch](https://github.com/apache/nifi/blob/master/nifi-commons/nifi-utils/src/main/java/org/apache/nifi/util/StopWatc
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102486824
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "JSON", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides JSON as FlowFile content "
++ "and individual attributes as FlowFile attributes. The
attributes are named as . "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final String APPLICATION_JSON = "application/json";
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private List timingStats = new ArrayList(); // stores
timing statistics
+private Map attributes = new TreeMap(); // stores CDA attributes
--- End diff --
This should be local to onTrigger(). I know you're using it in addElemen
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102492802
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "JSON", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides JSON as FlowFile content "
++ "and individual attributes as FlowFile attributes. The
attributes are named as . "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final String APPLICATION_JSON = "application/json";
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private List timingStats = new ArrayList(); // stores
timing statistics
+private Map attributes = new TreeMap(); // stores CDA attributes
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlC
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102493388
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "JSON", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides JSON as FlowFile content "
++ "and individual attributes as FlowFile attributes. The
attributes are named as . "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final String APPLICATION_JSON = "application/json";
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private List timingStats = new ArrayList(); // stores
timing statistics
+private Map attributes = new TreeMap(); // stores CDA attributes
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlC
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102494720
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "JSON", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides JSON as FlowFile content "
++ "and individual attributes as FlowFile attributes. The
attributes are named as . "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final String APPLICATION_JSON = "application/json";
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private List timingStats = new ArrayList(); // stores
timing statistics
+private Map attributes = new TreeMap(); // stores CDA attributes
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlC
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102498955
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "JSON", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides JSON as FlowFile content "
++ "and individual attributes as FlowFile attributes. The
attributes are named as . "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final String APPLICATION_JSON = "application/json";
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private List timingStats = new ArrayList(); // stores
timing statistics
+private Map attributes = new TreeMap(); // stores CDA attributes
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlC
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102500181
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "JSON", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides JSON as FlowFile content "
++ "and individual attributes as FlowFile attributes. The
attributes are named as . "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final String APPLICATION_JSON = "application/json";
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private List timingStats = new ArrayList(); // stores
timing statistics
+private Map attributes = new TreeMap(); // stores CDA attributes
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlC
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102508342
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "JSON", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides JSON as FlowFile content "
++ "and individual attributes as FlowFile attributes. The
attributes are named as . "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final String APPLICATION_JSON = "application/json";
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private List timingStats = new ArrayList(); // stores
timing statistics
+private Map attributes = new TreeMap(); // stores CDA attributes
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlC
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102509370
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "JSON", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides JSON as FlowFile content "
++ "and individual attributes as FlowFile attributes. The
attributes are named as . "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final String APPLICATION_JSON = "application/json";
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private List timingStats = new ArrayList(); // stores
timing statistics
+private Map attributes = new TreeMap(); // stores CDA attributes
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlC
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r102510825
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+import com.google.gson.Gson;
+import com.google.gson.GsonBuilder;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "JSON", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides JSON as FlowFile content "
++ "and individual attributes as FlowFile attributes. The
attributes are named as . "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final String APPLICATION_JSON = "application/json";
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private List timingStats = new ArrayList(); // stores
timing statistics
+private Map attributes = new TreeMap(); // stores CDA attributes
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlC
[GitHub] nifi issue #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1312 @kedarchitale I left some comments on some things that we need to change. Most are small but important modifications, but there's one thing that after some thought I think the processor is doing that it shouldn't: it's not the right place for transforming the output to JSON, especially since we have AttributesToJSON which will make it easy to do the same thing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request: https://github.com/apache/nifi/pull/1312#discussion_r104680681 --- Diff: nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-nar/pom.xml --- @@ -0,0 +1,40 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-ccda-bundle +1.2.0-SNAPSHOT + + +nifi-ccda-nar +nar + +true +true + + + + +org.apache.nifi +nifi-ccda-processors +1.2.0-SNAPSHOT --- End diff -- Let's remove this and add a dependencyManagement section to the CCDA root pom.xml --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request: https://github.com/apache/nifi/pull/1312#discussion_r104680946 --- Diff: nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-nar/src/main/resources/META-INF/LICENSE --- @@ -0,0 +1,238 @@ + + Apache License + Version 2.0, January 2004 +http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made,
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r104681118
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.util.StopWatch;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides individual attributes "
++ "as FlowFile attributes. The attributes are named as
. "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlContext jexlCtx = null; // JEXL Context to hold element
being processed
+
+private List properties;
+private Set relationships;
+
+/**
+ * SKIP_VALIDATION - Indicates whether to validate the CDA document
after loading.
+ * if true and the document is not valid, then ProcessException will
be thrown
+ */
+public static final PropertyDescrip
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r104681331
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.util.StopWatch;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides individual attributes "
++ "as FlowFile attributes. The attributes are named as
. "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlContext jexlCtx = null; // JEXL Context to hold element
being processed
+
+private List properties;
+private Set relationships;
+
+/**
+ * SKIP_VALIDATION - Indicates whether to validate the CDA document
after loading.
+ * if true and the document is not valid, then ProcessException will
be thrown
+ */
+public static final PropertyDescrip
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r104681504
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.util.StopWatch;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides individual attributes "
++ "as FlowFile attributes. The attributes are named as
. "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlContext jexlCtx = null; // JEXL Context to hold element
being processed
+
+private List properties;
+private Set relationships;
+
+/**
+ * SKIP_VALIDATION - Indicates whether to validate the CDA document
after loading.
+ * if true and the document is not valid, then ProcessException will
be thrown
+ */
+public static final PropertyDescrip
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r104681755
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.util.StopWatch;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides individual attributes "
++ "as FlowFile attributes. The attributes are named as
. "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlContext jexlCtx = null; // JEXL Context to hold element
being processed
+
+private List properties;
+private Set relationships;
+
+/**
+ * SKIP_VALIDATION - Indicates whether to validate the CDA document
after loading.
+ * if true and the document is not valid, then ProcessException will
be thrown
+ */
+public static final PropertyDescrip
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request: https://github.com/apache/nifi/pull/1312#discussion_r104681870 --- Diff: nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/resources/docs/org.apache.nifi.processors.ccda.ExtractCCDAAttributes/additionalDetails.html --- @@ -0,0 +1,63 @@ + + + + + +ExtractCCDAAttributes + + + + + +Apache NiFi C-CDA Processor + +C-CDA Processor Bundle provides parser for Consolidated-CDA documents +This processor parses C-CDA and sets attributes as FlowFile attributes. The attributes are named as <Parent> <dot> <Key>. +If the Parent is repeating, the naming will be <Parent> <underscore> <Parent Index> <dot> <Key>. + +Example Attribute Output + + +problemSection.act.observation.problemStatus.code.code=33999-4 + problemSection.act.observation.problemStatus.code.codeSystem=2.16.840.1.113883.6.1 +problemSection.act.observation.problemStatus.code.codeSystemName=LOINC +problemSection.act.observation.problemStatus.code.displayName=Status +problemSection.act.observation.problemStatus.statusCode.code=completed +problemSection.act.observation.statusCode.code=completed + + +Example Parser Mapping + +This processor is driven by a mapping file which specifies the element relationships. For example + + org.openhealthtools.mdht.uml.cda.consol.impl.ProblemObservationImpl=id#element.ids\ +@values#element.values[0]\ +@statusCode#element.statusCode\ +@effectiveTime#element.effectiveTime\ +@negation#element.negationInd\ +@problemStatus#element.problemStatus + org.openhealthtools.mdht.uml.cda.consol.impl.ProblemStatusImpl=id#element.id\ +@code#element.code\ +@values#element.values[0]\ +@statusCode#element.statusCode + + +References + +These mappings are defined as per the implementation guide -http://www.hl7.org/documentcenter/public/standards/dstu/CDAR2_IG_IHE_CONSOL_DSTU_R1dot1_2012JUL.zip";>HL7 Implementation Guide for CDA® Release 2 --- End diff -- Should use the mdash and reg HTML entities in this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1312 @kedarchitale We're getting pretty close. In addition to the per-line comments I noticed that you have DOS-style CRLF endings. Can you changes these to newlines only using dos2unix or something similar? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1312: NIFI-3147 CCDA Processor
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1312#discussion_r104701851
--- Diff:
nifi-nar-bundles/nifi-ccda-bundle/nifi-ccda-processors/src/main/java/org/apache/nifi/processors/ccda/ExtractCCDAAttributes.java
---
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.ccda;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Properties;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.jexl3.JexlBuilder;
+import org.apache.commons.jexl3.JexlContext;
+import org.apache.commons.jexl3.JexlEngine;
+import org.apache.commons.jexl3.JexlExpression;
+import org.apache.commons.jexl3.MapContext;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.util.StopWatch;
+import org.eclipse.emf.common.util.Diagnostic;
+import org.openhealthtools.mdht.uml.cda.CDAPackage;
+import org.openhealthtools.mdht.uml.cda.ClinicalDocument;
+import org.openhealthtools.mdht.uml.cda.ccd.CCDPackage;
+import org.openhealthtools.mdht.uml.cda.consol.ConsolPackage;
+import org.openhealthtools.mdht.uml.cda.hitsp.HITSPPackage;
+import org.openhealthtools.mdht.uml.cda.ihe.IHEPackage;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil;
+import org.openhealthtools.mdht.uml.cda.util.CDAUtil.ValidationHandler;
+
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"CCDA", "healthcare", "extract", "attributes"})
+@CapabilityDescription("Extracts information from an Consolidated CDA
formatted FlowFile and provides individual attributes "
++ "as FlowFile attributes. The attributes are named as
. "
++ "If the Parent is repeating, the naming will be
. "
++ "For example, section.act_07.observation.name=Essential
hypertension")
+public class ExtractCCDAAttributes extends AbstractProcessor {
+
+private static final char FIELD_SEPARATOR = '@';
+private static final char KEY_VALUE_SEPARATOR = '#';
+
+private Map> processMap = new
LinkedHashMap>(); // stores mapping data for Parser
+private JexlEngine jexl = null; // JEXL Engine to execute code for
mapping
+private JexlContext jexlCtx = null; // JEXL Context to hold element
being processed
+
+private List properties;
+private Set relationships;
+
+/**
+ * SKIP_VALIDATION - Indicates whether to validate the CDA document
after loading.
+ * if true and the document is not valid, then ProcessException will
be thrown
+ */
+public static final PropertyDescrip
[GitHub] nifi issue #1290: Fix NIFI-3142: ExtractHL7Attribute processor: Route a flow...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1290 @klinvill Do you know what version of NiFi you were running against? The NumberFormatException error was fixed in 1.1.0 by [NIFI-2887](https://issues.apache.org/jira/browse/NIFI-2887) and a relevant segment added to the unit tests. [NIFI-3142](https://issues.apache.org/jira/browse/NIFI-3142) suggests you ran into this against 1.0.0, and given the time you reported it at NiFi 1.1.0 had only been out for a short time so I wouldn't be surprised if you hadn't upgraded yet. Is this right? As for the NPE, I'll see if I can reproduce it but I don't know if I exactly follow the scenario. Is it an unrecognized segment name as in it's just bad HL7 or a segment that's unsupported for a particular version? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #854: NIFI-2564 Add support for HL7 composite component names
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/854 @joewitt It's rebased against master again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1584: NIFI-3585: Add DatabaseAdapter impl for MS SQL 2008
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1584#discussion_r105506187
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/impl/MSSQL2008DatabaseAdapter.java
---
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard.db.impl;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.processors.standard.db.DatabaseAdapter;
+
+/**
+ * A database adapter that generates MS SQL Compatible SQL for version
2008.
+ */
+public class MSSQL2008DatabaseAdapter implements DatabaseAdapter {
+@Override
+public String getName() {
+return "MS SQL 2008";
+}
+
+@Override
+public String getDescription() {
+return "Generates MS SQL Compatible SQL for version 2008";
+}
+
+@Override
+public String getSelectStatement(String tableName, String columnNames,
String whereClause, String orderByClause, Long limit, Long offset) {
+if (StringUtils.isEmpty(tableName)) {
+throw new IllegalArgumentException("Table name cannot be null
or empty");
+}
+
+final StringBuilder query = new StringBuilder("SELECT ");
+
+// If this is a limit query and not a paging query then use TOP in
MS SQL
+if (limit != null) {
+
+if (offset != null) {
+query.append("* FROM (SELECT ");
+}
+final long effectiveOffset = (offset == null) ? 0 : offset;
+if (effectiveOffset + limit > 0) {
+query.append("TOP ");
+query.append(effectiveOffset + limit);
+query.append(" ");
+}
+}
+
+if (StringUtils.isEmpty(columnNames) ||
columnNames.trim().equals("*")) {
+query.append("*");
+} else {
+query.append(columnNames);
+}
+
+if (limit != null && offset != null) {
+query.append(", ROW_NUMBER() OVER(ORDER BY ID asc) rnum");
--- End diff --
@mattyb149 I don't think there's a guarantee that the `ORDBER BY` column is
`ID`. You meant for this to be `orderByClause`, right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1584: NIFI-3585: Add DatabaseAdapter impl for MS SQL 2008
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1584 @mattyb149 I think I'll be able to say LGTM if this `orderByClause` thing gets fixed. Right now I think it only works on tables where the max value column is ID. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1584: NIFI-3585: Add DatabaseAdapter impl for MS SQL 2008
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1584 @mattyb149 Ok LGTM now so +1 I tested with [NIFI-3585.xml](https://gist.github.com/jfrazee/63fc76f25bc5cdcfaf66d246cf232d19) on an Amazon RDS SQL Server instance using the Orders table + OrderID column in the Northwind database [1,2,3]. 1. https://northwinddatabase.codeplex.com 2. http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Procedural.Importing.html#SQLServer.Procedural.Importing.Native.Using.Restore 3. https://aws.amazon.com/blogs/database/migrating-microsoft-sql-server-enterprise-workloads-to-amazon-rds-part-1/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1015: NIFI-2767 Periodically reload properties from file-...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1015 NIFI-2767 Periodically reload properties from file-based variable registry properties files You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-2767 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1015.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1015 commit 62ca8d2b2eb3bf71b83597eac80481d1b5539043 Author: Joey Frazee Date: 2016-09-13T15:00:13Z NIFI-2767 Periodically reload properties from file-based variable registry properties files --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1026: NIFI-2789 Add JMS properties to FlowFile attributes...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1026 NIFI-2789 Add JMS properties to FlowFile attributes on receive in ConsumeJMS You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-2789 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1026.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1026 commit d59b4338a838dff88c66ff86ea377071cf981765 Author: Joey Frazee Date: 2016-09-19T13:18:30Z Read JMS properties and add to FlowFile attributes in ConsumeJMS --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1027: NIFI-2790 Set JMS destination name on send/receive ...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1027 NIFI-2790 Set JMS destination name on send/receive instead of using the default destination Note: This was branched off of NIFI-2789 / #1026. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-2790 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1027.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1027 commit d59b4338a838dff88c66ff86ea377071cf981765 Author: Joey Frazee Date: 2016-09-19T13:18:30Z Read JMS properties and add to FlowFile attributes in ConsumeJMS commit 621fec64bd6ffbe7aefffcfd005421a6515c8ff2 Author: Joey Frazee Date: 2016-09-19T14:49:07Z Remove unused assertEquals import commit 1c76e22b06bd999496e19cd05d08db50660c1652 Author: Joey Frazee Date: 2016-09-19T15:50:59Z Move destination from default to send/receive to support EL better --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1026: NIFI-2789 Add JMS properties to FlowFile attributes...
Github user jfrazee closed the pull request at: https://github.com/apache/nifi/pull/1026 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1026: NIFI-2789 Add JMS properties to FlowFile attributes on rec...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1026 Closing in favor of #1027 which includes the properties enhancement too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1123: NIFI-2887 NumberFormatException in HL7ExtractAttrib...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1123 NIFI-2887 NumberFormatException in HL7ExtractAttributes for repeating segments with order control codes Fixes NIFI-2887 and adds more test cases You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-2887 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1123.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1123 commit 672bd4f4f5767b80c57fc836e89051a1aa048db1 Author: Joey Frazee Date: 2016-10-11T15:10:43Z Fixed HL7 segment index bug for order control codes, more testing commit 2d1555d09c11f5a3a13f5f61e3eb0c30b5e5a895 Author: Joey Frazee Date: 2016-10-11T15:43:49Z Added NOTICE entries for test data and fixed checkstyle violations --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1137: NIFI-1710 - Resolve path name to nifi.sh script
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1137 @trixpan Thanks for picking this up again! Since this is just about variation in the arguments that readlink supports or the availability of realpath, I would have thought it'd just be simpler to have a case statement around the platform/uname. I don't see anything wrong with it at all, it's just a little complicated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #854: NIFI-2564 Add support for HL7 composite component names
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/854 @trixpan I'll be getting back to this today. There was a bug fix that had to slide in before this too so it's not going to rebase cleanly anymore. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1250: NIFI-3061 PutAzureEventHub blocks after network int...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1250 NIFI-3061 PutAzureEventHub blocks after network interruption This change maintains status quo and does not enable the existing unit tests. The change was tested against [NIFI-3061.xml](https://gist.github.com/jfrazee/6820ec854974df88519f99b3388a0874) though. As mentioned in [NIFI-3070](https://issues.apache.org/jira/browse/NIFI-3070), there isn't a good way to provide meaningful unit tests so integration tests should be added as a new improvement. - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-3061 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1250.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1250 commit e67548fb804fbfd35dcd77886d47a0c38e0353c0 Author: Joey Frazee Date: 2016-11-19T21:25:19Z Bumped azure-eventhubs version to 0.9.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1198: [NIFI-3015] Run NiFi service from nifi user instead of roo...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1198 @fidget83 I think the right thing is to find some way to update the run.as setting, as @trixpan mentions, from the rpm since we know the nifi user exists then and fold that into the rpm spec. Right now, `nifi.sh install` can and does get used on non-RedHat derived platforms without rpms and in tarball installs so there's no guarantee the user exists. It's a good change if done right though. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1273: NIFI-3104 Remove MPL-licensed test data from TestEx...
GitHub user jfrazee opened a pull request: https://github.com/apache/nifi/pull/1273 NIFI-3104 Remove MPL-licensed test data from TestExtractHL7Attributes This PR removes the offending tests and updates relevant NOTICE files to remove, now irrelevant, notices. - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [X] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? You can merge this pull request into a Git repository by running: $ git pull https://github.com/jfrazee/nifi NIFI-3104 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1273.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1273 commit a60c2197f96ac2182d0b5a5809deab19be5f9251 Author: Joey Frazee Date: 2016-11-25T23:39:09Z Removed tests derived from HAPI test data --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1934: NIFI-4106 Lower loglevel of authentication request and suc...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1934 @NielsZeilemaker I looked at this and that was exactly my concern as well and I think your suggestion of inverting the change to make the rejection WARN is a good one. You get the control over successes via the level for the class but you won't lose the rejection which the defaults really should include to not undermine security. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1696: NIFI-1655 - Add .gitattributes to specifically define
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1696 @trixpan I think there's a mistake in the last line of .gitattributes. It is missing the 'text' qualifier and just is the file pattern and then eol. Let me know if I've misunderstood; otherwise I'll +1 and I can fix it during the merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1953: NIFI-4130 Add lookup controller service in TransformXML to...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1953 @pvillard31 It seems like we should just use LookupAttribute for this use case. Thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1953: NIFI-4130 Add lookup controller service in TransformXML to...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1953 @pvillard31 Right, wouldn't want to do that. I misunderstood what you were doing. This makes sense. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1969: NIFI-4082 - Added EL on GetMongo properties
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1969 @pvillard31 I think we can add EL to collection without having to do anything fancy since getCollection() in AbstractMongoProcessor re-uses the mongoClient already. That said, is it worth also adding EL support for URI and DB in this PR? The simple case will mean it'll get eval'd against the variable registry, which is definitely useful for migrating from env to env. There's another case though in being able to eval against a FlowFile for PutMongo so you can route to different DBs if you're doing some application-level sharding. But that would mean we'd have to manage a pool/map of connections since the mongoClient currently is set with @OnScheduled. Makes sense if you think that's for another PR though. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1969: NIFI-4082 - Added EL on GetMongo properties
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1969 @pvillard31 I left some comments on [NIFI-4149](https://issues.apache.org/jira/browse/NIFI-4149). I think it'd be a step in the right direction. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1969: NIFI-4082 - Added EL on GetMongo properties
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1969#discussion_r127808068
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/GetMongo.java
---
@@ -59,14 +59,21 @@
public static final Validator DOCUMENT_VALIDATOR = new Validator() {
@Override
public ValidationResult validate(final String subject, final
String value, final ValidationContext context) {
+final ValidationResult.Builder builder = new
ValidationResult.Builder();
+builder.subject(subject).input(value);
+
+if (context.isExpressionLanguageSupported(subject) &&
context.isExpressionLanguagePresent(value)) {
+return builder.valid(true).explanation("Contains
Expression Language").build();
+}
+
String reason = null;
try {
Document.parse(value);
} catch (final RuntimeException e) {
reason = e.getClass().getName();
--- End diff --
Seems like this should be `e.getMessage()` or `get.getClass().getName() +
": " + e.getMessage()` if you want the underlying exception class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1969: NIFI-4082 - Added EL on GetMongo properties
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1969 @pvillard31 I think this needs more updates to the tests. Since you've added URI, collection, and DB, we should probably add that to the Get test. And, I think there are similar changes that make sense for the Put and Abstract processor. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2155: NIFI-4193 Converting to use the dockerfile-maven plugin
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/2155 Reviewing for once ---
