[GitHub] nifi-minifi-cpp issue #168: MINIFICPP-280 Move extension tests and test head...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/168 @phrocker Sure thing. I'm not with the project that needed minifi anymore, but I'm still interested in seeing these changes implemented. ---
[GitHub] nifi-minifi-cpp issue #191: MINIFICPP-114 Consolidate JSON API use to RapidJ...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/191 Squashed. @phrocker No, I didn't create a benchmark. The slowdown is surprising, though. I can't prove it, but I think larger production payloads will show improvement. ---
[GitHub] nifi-minifi-cpp issue #191: MINIFICPP-114 Consolidate JSON API use to RapidJ...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/191 @phrocker Honestly, I just reimplemented the JSON handling in RapidJSON as well as I could, and fixed the things that broke the build and tests. Help would be greatly appreciated. ---
[GitHub] nifi-minifi-cpp issue #185: MINIFICPP-303 Upgrade civetweb and rocksdb
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/185 I see it in df353561c8a8a0a0d1178a76f03b966b44542834. Hopefully that commit hash will resolve once the sync is complete. Thanks! ---
[GitHub] nifi-minifi-cpp pull request #185: MINIFICPP-303 Upgrade civetweb and rocksd...
Github user calebj closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/185 ---
[GitHub] nifi-minifi-cpp issue #185: MINIFICPP-303 Upgrade civetweb and rocksdb
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/185 I did. The new commit messages reflect the new version, too. ---
[GitHub] nifi-minifi-cpp issue #157: MINIFICPP-269 Implement ApplyTemplate processor
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/157 @phrocker Sure, I'll do that when I get a chance to. ---
[GitHub] nifi-minifi-cpp pull request #157: MINIFICPP-269 Implement ApplyTemplate pro...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/157#discussion_r153602526
--- Diff: CMakeLists.txt ---
@@ -173,6 +173,12 @@ if (ENABLE_TENSORFLOW)
createExtension(TENSORFLOW-EXTENSIONS "TENSORFLOW EXTENSIONS" "This
enables TensorFlow support" "extensions/tensorflow"
"${TEST_DIR}/tensorflow-tests")
endif()
+## Bustache/template extensions
+option(ENABLE_BUSTACHE "Enables Bustache (ApplyTemplate) support." OFF)
--- End diff --
Ah, my bad. Fixing that uncovered a missing include, but both issues are
resolved in the latest push.
---
[GitHub] nifi-minifi-cpp pull request #157: MINIFICPP-269 Implement ApplyTemplate pro...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/157#discussion_r153566702
--- Diff: CMakeLists.txt ---
@@ -173,6 +173,12 @@ if (ENABLE_TENSORFLOW)
createExtension(TENSORFLOW-EXTENSIONS "TENSORFLOW EXTENSIONS" "This
enables TensorFlow support" "extensions/tensorflow"
"${TEST_DIR}/tensorflow-tests")
endif()
+## Bustache/template extensions
+option(ENABLE_BUSTACHE "Enables Bustache (ApplyTemplate) support." OFF)
--- End diff --
What with?
---
[GitHub] nifi-minifi-cpp issue #185: MINIFICPP-303 Upgrade civetweb and rocksdb
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/185 See if it works after the next push. I'm putting 5.8.6 in afresh and disabling the tests in a cleaner fashion. ---
[GitHub] nifi-minifi-cpp issue #185: MINIFICPP-303 Upgrade civetweb and rocksdb
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/185 Weird. Nothing in the process should have changed. Where is that happening? ---
[GitHub] nifi-minifi-cpp issue #206: MINIFICPP-320: Change Processors to be within th...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/206 This causes some tests to fail to link on Linux both on Travis CI and cloud9, both of which are Ubuntu Trusty. Here's the output: ``` [ 19%] Linking CXX executable TestExecuteProcess libminifi/libminifi.a(ListenHTTP.cpp.o): In function `org::apache::nifi::minifi::processors::ListenHTTP::WriteCallback::process(std::shared_ptr)': ListenHTTP.cpp:(.text+0x9e): undefined reference to `mg_read' ListenHTTP.cpp:(.text+0x10e): undefined reference to `mg_read' libminifi/libminifi.a(ListenHTTP.cpp.o): In function `org::apache::nifi::minifi::processors::ListenHTTP::Handler::handlePost(CivetServer*, mg_connection*)': ListenHTTP.cpp:(.text+0xd80): undefined reference to `mg_get_request_info' ListenHTTP.cpp:(.text+0xe50): undefined reference to `mg_printf' ListenHTTP.cpp:(.text+0x133d): undefined reference to `mg_printf' ListenHTTP.cpp:(.text+0x14c8): undefined reference to `mg_printf' ListenHTTP.cpp:(.text+0x1508): undefined reference to `mg_printf' ListenHTTP.cpp:(.text+0x15a4): undefined reference to `mg_printf' libminifi/libminifi.a(ListenHTTP.cpp.o):ListenHTTP.cpp:(.text+0x16a6): more undefined references to `mg_printf' follow libminifi/libminifi.a(ListenHTTP.cpp.o): In function `org::apache::nifi::minifi::processors::ListenHTTP::onSchedule(org::apache::nifi::minifi::core::ProcessContext*, org::apache::nifi::minifi::core::ProcessSessionFactory*)': ListenHTTP.cpp:(.text+0x254a): undefined reference to `CivetServer::CivetServer(std::vector >, CivetCallbacks const*)' ListenHTTP.cpp:(.text+0x25cb): undefined reference to `CivetServer::addHandler(std::string const&, CivetHandler*)' libminifi/libminifi.a(ListenHTTP.cpp.o): In function `org::apache::nifi::minifi::processors::ListenHTTP::Handler::sendErrorResponse(mg_connection*)': ListenHTTP.cpp:(.text+0x76b): undefined reference to `mg_printf' libminifi/libminifi.a(ListenHTTP.cpp.o):(.rodata._ZTIN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE[_ZTIN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE]+0x10): undefined reference to `typeinfo for CivetHandler' libminifi/libminifi.a(ListenHTTP.cpp.o):(.rodata._ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE[_ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE]+0x20): undefined reference to `CivetHandler::handleGet(CivetServer*, mg_connection*)' libminifi/libminifi.a(ListenHTTP.cpp.o):(.rodata._ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE[_ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE]+0x30): undefined reference to `CivetHandler::handleHead(CivetServer*, mg_connection*)' libminifi/libminifi.a(ListenHTTP.cpp.o):(.rodata._ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE[_ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE]+0x38): undefined reference to `CivetHandler::handlePut(CivetServer*, mg_connection*)' libminifi/libminifi.a(ListenHTTP.cpp.o):(.rodata._ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE[_ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE]+0x40): undefined reference to `CivetHandler::handleDelete(CivetServer*, mg_connection*)' libminifi/libminifi.a(ListenHTTP.cpp.o):(.rodata._ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE[_ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE]+0x48): undefined reference to `CivetHandler::handleOptions(CivetServer*, mg_connection*)' libminifi/libminifi.a(ListenHTTP.cpp.o):(.rodata._ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE[_ZTVN3org6apache4nifi6minifi10processors10ListenHTTP7HandlerE]+0x50): undefined reference to `CivetHandler::handlePatch(CivetServer*, mg_connection*)' collect2: error: ld returned 1 exit status make[2]: *** [TestExecuteProcess] Error 1 make[1]: *** [CMakeFiles/TestExecuteProcess.dir/all] Error 2 make: *** [all] Error 2 ``` ---
[GitHub] nifi-minifi-cpp issue #203: MINIFICPP-251 Move Civet implementations to an e...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/203 Should be fine now, I think. ---
[GitHub] nifi-minifi-cpp issue #203: MINIFICPP-251 Move Civet implementations to an e...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/203 Still some issues to work out. ---
[GitHub] nifi-minifi-cpp pull request #203: MINIFICPP-251 Move Civet implementations ...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/203 MINIFICPP-251 Move Civet implementations to an extension. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp MINIFICPP-251 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/203.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #203 commit ae09003d288bf9a7fc9b9832574a17154f29ee9a Author: Caleb Johnson Date: 2017-11-16T23:06:22Z MINIFICPP-251 Move Civet implementations to an extension. ---
[GitHub] nifi-minifi-cpp issue #199: MINIFICPP-316: Fix GPS tests and resolve linter ...
Github user calebj commented on the issue:
https://github.com/apache/nifi-minifi-cpp/pull/199
I'm too late, but removing the `CATCH_CONFIG_MAIN` and linking to
`${CATCH_MAIN_LIB}` instead would be mode consistent with the other extensions'
test creation script. I'll put that into #168 for now.
---
[GitHub] nifi-minifi-cpp issue #198: Fix travis-breaking linter error in TensorFlowTe...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/198 Superceded by #199 ---
[GitHub] nifi-minifi-cpp pull request #198: Fix travis-breaking linter error in Tenso...
Github user calebj closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/198 ---
[GitHub] nifi-minifi-cpp pull request #198: Fix travis-breaking linter error in Tenso...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/198 Fix travis-breaking linter error in TensorFlowTests.cpp Is a JIRA ticket necessary for this? It's literally a difference of one character. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp TensorFlowTestsLint Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/198.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #198 commit f89ad47cd4a3e5ae6d842fbb547002518120ac3e Author: Caleb Johnson Date: 2017-11-21T13:59:10Z Fix travis-breaking linter error in TensorFlowTests.cpp ---
[GitHub] nifi-minifi-cpp pull request #191: MINIFICPP-114 Consolidate JSON API use to...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r152182652
--- Diff: extensions/libarchive/UnfocusArchiveEntry.cpp ---
@@ -67,55 +66,41 @@ void
UnfocusArchiveEntry::onTrigger(core::ProcessContext *context, core::Process
}
fileutils::FileManager file_man;
-
- // Get lens stack from attribute
ArchiveMetadata lensArchiveMetadata;
- Json::Value lensStack;
- Json::Reader reader;
-
- std::string existingLensStack;
-
- if (flowFile->getAttribute("lens.archive.stack", existingLensStack)) {
-logger_->log_info("UnfocusArchiveEntry loading existing lens context");
-
-// TODO(calebj) handle any exceptions that might arise from working
with JSON data
-if (!reader.parse(existingLensStack, lensStack)) {
- logger_->log_error("UnfocusArchiveEntry JSON parse error: %s",
reader.getFormattedErrorMessages());
- context->yield();
- return;
+ // Get lens stack from attribute
+ {
+ArchiveStack archiveStack;
+{
+ std::string existingLensStack;
+
+ if (flowFile->getAttribute("lens.archive.stack", existingLensStack))
{
+logger_->log_info("FocusArchiveEntry loading existing lens
context");
+
+try {
+ archiveStack.loadJsonString(existingLensStack);
--- End diff --
Done in 00282396e2b2fa3f0f450a7916d692b52c749a1d
---
[GitHub] nifi-minifi-cpp pull request #191: MINIFICPP-114 Consolidate JSON API use to...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r152180661
--- Diff: libminifi/src/RemoteProcessorGroupPort.cpp ---
@@ -264,20 +264,27 @@ void
RemoteProcessorGroupPort::refreshRemoteSite2SiteInfo() {
if (!response_body.empty()) {
std::string controller = std::string(response_body.begin(),
response_body.end());
logger_->log_debug("controller config %s", controller.c_str());
- Json::Value value;
- Json::Reader reader;
- bool parsingSuccessful = reader.parse(controller, value);
- if (parsingSuccessful && !value.empty()) {
-Json::Value controllerValue = value["controller"];
-if (!controllerValue.empty()) {
- Json::Value port = controllerValue["remoteSiteListeningPort"];
- if (client_type_ == sitetosite::CLIENT_TYPE::RAW &&
!port.empty())
-this->site2site_port_ = port.asInt();
+
--- End diff --
All I did here was translate the jsoncpp code to rapidjson. I didn't test
it.
---
[GitHub] nifi-minifi-cpp issue #191: MINIFICPP-114 Consolidate JSON API use to RapidJ...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/191 I know that it's MIT licensed, but their license.txt comes with others tacked on for subcomponents. If you think they're all compatible, I'll copy all of them in. ---
[GitHub] nifi-minifi-cpp issue #191: MINIFICPP-114 Consolidate JSON API use to RapidJ...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/191 @achristianson something in the master branch is failing. Looks like something in the commit before the latest one, 9b9c3354330d525cfdd50f656ae42fc3da80764c, broke it. Regarding the LICENSE, there are a handful of them. IANAL, so can someone look over [rapidjson's license.txt](https://github.com/Tencent/rapidjson/blob/master/license.txt) and see what might have to be left out, if anything? ---
[GitHub] nifi-minifi-cpp pull request #191: MINIFICPP-114 Consolidate JSON API use to...
Github user calebj commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151581577 --- Diff: extensions/http-curl/protocols/RESTReceiver.cpp --- @@ -65,38 +65,8 @@ void RESTReceiver::initialize(const std::shared_ptr
[GitHub] nifi-minifi-cpp pull request #191: MINIFICPP-114 Consolidate JSON API use to...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151581091
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const
C2Payload &payload, const
return std::move(C2Payload(payload.getOperation(),
state::UpdateState::READ_ERROR, true));
}
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value &json_root,
const C2Payload &payload) {
- // get the name from the content
- Json::Value json_payload;
- std::map> children;
- for (const auto &nested_payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload,
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
- }
- for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
- Json::Value children_json(Json::arrayValue);
- for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
- }
-} else {
- if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
- json_payload[child_vector.first] =
child_vector.second.at(0)[child_vector.first];
-} else {
- json_payload[child_vector.first] = child_vector.second.at(0);
-}
- }
-}
- }
+void setJsonStr(const std::string& key, const std::string& value,
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
--- End diff --
The linter wants a constant ref passed, but it doesn't compile due to
rapidjson's semantics.
---
[GitHub] nifi-minifi-cpp pull request #191: MINIFICPP-114 Consolidate JSON API use to...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151580773
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const
C2Payload &payload, const
return std::move(C2Payload(payload.getOperation(),
state::UpdateState::READ_ERROR, true));
}
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value &json_root,
const C2Payload &payload) {
- // get the name from the content
- Json::Value json_payload;
- std::map> children;
- for (const auto &nested_payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload,
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
- }
- for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
- Json::Value children_json(Json::arrayValue);
- for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
- }
-} else {
- if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
- json_payload[child_vector.first] =
child_vector.second.at(0)[child_vector.first];
-} else {
- json_payload[child_vector.first] = child_vector.second.at(0);
-}
- }
-}
- }
+void setJsonStr(const std::string& key, const std::string& value,
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
+ rapidjson::Value keyVal;
+ rapidjson::Value valueVal;
+ const char* c_key = key.c_str();
+ const char* c_val = value.c_str();
+ keyVal.SetString(c_key, key.length()), alloc;
+ valueVal.SetString(c_val, value.length(), alloc);
+
+ parent.AddMember(keyVal, valueVal, alloc);
+}
+
+rapidjson::Value getStringValue(const std::string& value,
rapidjson::Document::AllocatorType& alloc) { // NOLINT
+ rapidjson::Value Val;
+ Val.SetString(value.c_str(), value.length(), alloc);
+ return Val;
+}
+
+void RESTProtocol::mergePayloadContent(rapidjson::Value &target, const
C2Payload &payload, rapidjson::Document::AllocatorType &alloc) {
const std::vector &content = payload.getContent();
+
for (const auto &payload_content : content) {
-Json::Value payload_content_values;
+rapidjson::Value payload_content_values(rapidjson::kObjectType);
bool use_sub_option = true;
+
if (payload_content.op == payload.getOperation()) {
for (auto content : payload_content.operation_arguments) {
if (payload_content.operation_arguments.size() == 1 &&
payload_content.name == content.first) {
- json_payload[payload_content.name] = content.second;
+ setJsonStr(payload_content.name, content.second, target, alloc);
use_sub_option = false;
} else {
- payload_content_values[content.first] = content.second;
+ setJsonStr(content.first, content.second,
payload_content_values, alloc);
}
}
}
-if (use_sub_option)
- json_payload[payload_content.name] = payload_content_values;
+if (use_sub_option) {
+ rapidjson::Value sub_key = getStringValue(payload_content.name,
alloc);
+ target.AddMember(sub_key, payload_content_values, alloc);
+}
+ }
+}
+
+std::string RESTProtocol::serializeJsonRootPayload(const C2Payload&
payload) {
+ rapidjson::Document json_payload(rapidjson::kObjectType);
+ rapidjson::Document::AllocatorType &alloc = json_payload.GetAllocator();
--- End diff --
I believe it's supposed to be per-document.
---
[GitHub] nifi-minifi-cpp pull request #191: MINIFICPP-114 Consolidate JSON API use to...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151580663
--- Diff:
libminifi/src/core/reporting/SiteToSiteProvenanceReportingTask.cpp ---
@@ -53,54 +56,92 @@ void SiteToSiteProvenanceReportingTask::initialize() {
RemoteProcessorGroupPort::initialize();
}
+void setJsonStr(const std::string& key, const std::string& value,
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
--- End diff --
It's just a shortcut wrapper for adding members from strings. I ran into a
lot of issues with strings being read as garbage. I'll move them to a central
location.
---
[GitHub] nifi-minifi-cpp pull request #191: MINIFICPP-114 Consolidate JSON API use to...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/191#discussion_r151580153
--- Diff: extensions/http-curl/protocols/RESTProtocol.cpp ---
@@ -81,48 +93,117 @@ const C2Payload RESTProtocol::parseJsonResponse(const
C2Payload &payload, const
return std::move(C2Payload(payload.getOperation(),
state::UpdateState::READ_ERROR, true));
}
-Json::Value RESTProtocol::serializeJsonPayload(Json::Value &json_root,
const C2Payload &payload) {
- // get the name from the content
- Json::Value json_payload;
- std::map> children;
- for (const auto &nested_payload : payload.getNestedPayloads()) {
-Json::Value child_payload = serializeJsonPayload(json_payload,
nested_payload);
-children[nested_payload.getLabel()].push_back(child_payload);
- }
- for (auto child_vector : children) {
-if (child_vector.second.size() > 1) {
- Json::Value children_json(Json::arrayValue);
- for (auto child : child_vector.second) {
-json_payload[child_vector.first] = child;
- }
-} else {
- if (child_vector.second.size() == 1) {
-if (child_vector.second.at(0).isMember(child_vector.first)) {
- json_payload[child_vector.first] =
child_vector.second.at(0)[child_vector.first];
-} else {
- json_payload[child_vector.first] = child_vector.second.at(0);
-}
- }
-}
- }
+void setJsonStr(const std::string& key, const std::string& value,
rapidjson::Value& parent, rapidjson::Document::AllocatorType& alloc) { // NOLINT
+ rapidjson::Value keyVal;
+ rapidjson::Value valueVal;
+ const char* c_key = key.c_str();
+ const char* c_val = value.c_str();
+ keyVal.SetString(c_key, key.length()), alloc;
+ valueVal.SetString(c_val, value.length(), alloc);
+
+ parent.AddMember(keyVal, valueVal, alloc);
+}
+
+rapidjson::Value getStringValue(const std::string& value,
rapidjson::Document::AllocatorType& alloc) { // NOLINT
+ rapidjson::Value Val;
+ Val.SetString(value.c_str(), value.length(), alloc);
+ return Val;
+}
+
+void RESTProtocol::mergePayloadContent(rapidjson::Value &target, const
C2Payload &payload, rapidjson::Document::AllocatorType &alloc) {
const std::vector &content = payload.getContent();
+
for (const auto &payload_content : content) {
-Json::Value payload_content_values;
+rapidjson::Value payload_content_values(rapidjson::kObjectType);
bool use_sub_option = true;
+
if (payload_content.op == payload.getOperation()) {
for (auto content : payload_content.operation_arguments) {
if (payload_content.operation_arguments.size() == 1 &&
payload_content.name == content.first) {
- json_payload[payload_content.name] = content.second;
+ setJsonStr(payload_content.name, content.second, target, alloc);
use_sub_option = false;
} else {
- payload_content_values[content.first] = content.second;
+ setJsonStr(content.first, content.second,
payload_content_values, alloc);
}
}
}
-if (use_sub_option)
- json_payload[payload_content.name] = payload_content_values;
+if (use_sub_option) {
+ rapidjson::Value sub_key = getStringValue(payload_content.name,
alloc);
+ target.AddMember(sub_key, payload_content_values, alloc);
+}
+ }
+}
+
+std::string RESTProtocol::serializeJsonRootPayload(const C2Payload&
payload) {
+ rapidjson::Document json_payload(rapidjson::kObjectType);
+ rapidjson::Document::AllocatorType &alloc = json_payload.GetAllocator();
+
+ rapidjson::Value opReqStrVal;
+ std::string operation_request_str = getOperation(payload);
+ opReqStrVal.SetString(operation_request_str.c_str(),
operation_request_str.length(), alloc);
+ json_payload.AddMember("operation", opReqStrVal, alloc);
+
+ std::string operationid = payload.getIdentifier();
+ if (operationid.length() > 0) {
+rapidjson::Value operationIdVal = getStringValue(operationid, alloc);
+json_payload.AddMember("operationid", operationIdVal, alloc);
+ }
+
+ mergePayloadContent(json_payload, payload, alloc);
+
+ for (const auto &nested_payload : payload.getNestedPayloads()) {
+rapidjson::Value np_key = getStringValue(nested_payload.getLabel(),
alloc);
+rapidjson::Value np_value = serializeJsonPayload(nested_payload,
alloc);
+json_payload.AddMember(np_key, np_value, alloc);
+ }
+
+ rapidjson::StringBuffer buffer;
+ rapidjson::PrettyWriter writer(buffer);
--- En
[GitHub] nifi-minifi-cpp pull request #191: MINIFICPP-114 Consolidate JSON API use to...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/191 MINIFICPP-114 Consolidate JSON API use to RapidJSON Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp MINIFICPP-114 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/191.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #191 commit cfe9421098f1ab3b09528b56f12482a3147db9cb Author: Caleb Johnson Date: 2017-11-15T12:38:54Z MINIFICPP-114 Consolidate JSON API use to RapidJSON ---
[GitHub] nifi-minifi-cpp issue #156: MINIFICPP-268 Implement ManipulateArchive proces...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/156 Should be ready now. Check to see if the exceptions and error output on misconfigurations fits what you want. ---
[GitHub] nifi-minifi-cpp issue #156: MINIFICPP-268 Implement ManipulateArchive proces...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/156 Is this waiting on #181? I have another branch based on that code I can push this PR if needed. Otherwise, is there an estimate as to when this might be merged? ---
[GitHub] nifi-minifi-cpp issue #157: MINIFICPP-269 Implement ApplyTemplate processor
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/157 Is there an estimate as to when this can be merged? ---
[GitHub] nifi-minifi-cpp pull request #185: MINIFICPP-303 Upgrade civetweb and rocksd...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/185 MINIFICPP-303 Upgrade civetweb and rocksdb Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp MINIFICPP-303 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/185.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #185 ---
[GitHub] nifi-minifi-cpp pull request #184: MINIFICPP-302 Correct USB camera createEx...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/184 MINIFICPP-302 Correct USB camera createExtension Configuration fails because the USB camera extension's createExtension in the top-level CMakeLists wasn't updated to match the changes to that macro in the GetGPS processor patch. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp MINIFICPP-302 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/184.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #184 commit 87f3c408f6b76d2148d429437ef4b32454ed8509 Author: Caleb Johnson Date: 2017-11-13T12:50:39Z MINIFICPP-302 Correct USB camera createExtension ---
[GitHub] nifi-minifi-cpp pull request #170: MINIFICPP-250: Initial implementation fo ...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/170#discussion_r150017488
--- Diff: .travis.yml ---
@@ -84,4 +84,4 @@ matrix:
- package='libpng'; [[ $(brew ls --versions ${package}) ]] && {
brew outdated ${package} || brew upgrade ${package}; } || brew install
${package}
script:
- - mkdir ./build && cd ./build && cmake .. ${CMAKE_BUILD_OPTIONS} && make
-j2 VERBOSE=1 && make test ARGS="-j2 --output-on-failure" && make linter &&
make apache-rat && make docs
+ - mkdir ./build && cd ./build && cmake -DENABLE_PCAP=TRUE ..
${CMAKE_BUILD_OPTIONS} && make -j2 VERBOSE=1 && make test ARGS="-j2
--output-on-failure" && make linter && make apache-rat && make docs
--- End diff --
I think this is better off in CMAKE_BUILD_OPTIONS above
---
[GitHub] nifi-minifi-cpp pull request #180: MINIFICPP-258 enable C2NullConfiguration ...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/180 MINIFICPP-258 enable C2NullConfiguration test Not entirely sure if this is correct; I'm just going off of what the spec was before this test was removed. It appears to run just fine on both the Linux and OSX environments. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp MINIFICPP-258 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/180.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #180 commit b9731206bd0201644b8a5ad99785abe927618d37 Author: Caleb Johnson Date: 2017-11-08T13:50:28Z MINIFICPP-258 add C2NullConfiguration ---
[GitHub] nifi-minifi-cpp pull request #169: MINIFICPP-282 Improve test build speed, a...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/169 MINIFICPP-282 Improve test build speed, add Travis build cache Split off from MINIFICPP-280 #168 as requested Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp MINIFICPP-282 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/169.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #169 commit 8271168877bb2b94c6357d5685511662e4f3640e Author: Caleb Johnson Date: 2017-11-02T13:03:28Z MINIFICPP-282 Improve test build speed, add Travis build cache ---
[GitHub] nifi-minifi-cpp pull request #157: MINIFICPP-269 Implement ApplyTemplate pro...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/157#discussion_r148581669
--- Diff: extensions/bustache/ApplyTemplate.cpp ---
@@ -0,0 +1,100 @@
+/**
+ * @file ApplyTemplate.cpp
+ * ApplyTemplate class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+
+#include
+#include
+
+#include "ApplyTemplate.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ApplyTemplate::Template("Template", "Path to the input
mustache template file", "");
+core::Relationship ApplyTemplate::Success("success", "success operational
on the flow record");
+
+void ApplyTemplate::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Template);
+setSupportedProperties(properties);
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ApplyTemplate::onTrigger(core::ProcessContext *context,
core::ProcessSession *session) {
+auto flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+std::string templateFile;
+context->getProperty(Template.getName(), templateFile);
+WriteCallback cb(templateFile, flowFile);
+session->write(flowFile, &cb);
+session->transfer(flowFile, Success);
+}
+
+ApplyTemplate::WriteCallback::WriteCallback(const std::string& path,
std::shared_ptr flowFile) {
+logger_ =
logging::LoggerFactory::getLogger();
+templateFile_ = path;
+flowFile_ = flowFile;
+}
+
+int64_t
ApplyTemplate::WriteCallback::process(std::shared_ptr stream) {
--- End diff --
It won't let me declare a different prototype than the virtual one in
OutputStreamCallback
```
[ 85%] Building CXX object
extensions/bustache/CMakeFiles/minifi-bustache-extensions.dir/ApplyTemplate.cpp.o
/home/ubuntu/workspace/extensions/bustache/ApplyTemplate.cpp: In member
function âvirtual void
org::apache::nifi::minifi::processors::ApplyTemplate::onTrigger(const
std::shared_ptr&, const
std::shared_ptr&)â:
/home/ubuntu/workspace/extensions/bustache/ApplyTemplate.cpp:66:19: error:
cannot declare variable âcbâ to be of abstract type
âorg::apache::nifi::minifi::processors::ApplyTemplate::WriteCallbackâ
WriteCallback cb(templateFile,
std::shared_ptr(flowFile));
^
In file included from
/home/ubuntu/workspace/extensions/bustache/ApplyTemplate.cpp:32:0:
/home/ubuntu/workspace/extensions/bustache/ApplyTemplate.h:61:11: note:
because the following virtual functions are pure within
âorg::apache::nifi::minifi::processors::ApplyTemplate::WriteCallbackâ:
class WriteCallback : public OutputStreamCallback {
^
In file included from
/home/ubuntu/workspace/extensions/bustache/../../libminifi/include/core/ProcessSession.h:31:0,
from
/home/ubuntu/workspace/extensions/bustache/../../libminifi/include/core/Processor.h:42,
from
/home/ubuntu/workspace/extensions/bustache/ApplyTemplate.h:25,
from
/home/ubuntu/workspace/extensions/bustache/ApplyTemplate.cpp:32:
/home/ubuntu/workspace/extensions/bustache/../../libminifi/include/FlowFileRecord.h:97:19:
note:virtual int64_t
org::apache::nifi::minifi::OutputStreamCallback::process(std::shared_ptr)
virtual int64_t process(std::shared_ptr stream) = 0;
^
make[2]: ***
[extensions/bustache/CMakeFiles/minifi-b
[GitHub] nifi-minifi-cpp pull request #168: MINIFICPP-280 Refactoring and various imp...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/168 MINIFICPP-280 Refactoring and various improvements - move extension tests into their respective folders - separate source and header files - remove unnecessary or nonexisting include directories - run linter on extension source files as part of linter target - clean up extensions according to linter - add ability to specify more than one include and source folder for linter - build catch main() and spdlib as shared objects for all tests (faster build!) - cmake doesn't know PRIVATE BEFORE, only BEFORE PRIVATE - borrow changes to tests from MINIFICPP-60 for parallel testing - enable parallel testing in travis config Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp ExtensionLint Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/168.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #168 commit 37d64e50a378a812970a7c468bf3cd4051ba6cf7 Author: Caleb Johnson Date: 2017-11-01T16:52:55Z MINIFICPP-280 Refactoring and various improvements - move extension tests into their respective folders - separate source and header files - remove unnecessary or nonexisting include directories - run linter on extension source files as part of linter target - clean up extensions according to linter - add ability to specify more than one include and source folder for linter - build catch main() and spdlib as shared objects for all tests (faster build!) - cmake doesn't know PRIVATE BEFORE, only BEFORE PRIVATE - borrow changes to tests from MINIFICPP-60 for parallel testing - enable parallel testing in travis config ---
[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r148094619
--- Diff: CMakeLists.txt ---
@@ -111,38 +111,38 @@ add_subdirectory(libminifi)
#function(createExtension extensionCondition extensionGuard extensionName
description dirName)
## Add http-curl extensions
-createExtension(DISABLE_CURL
- HTTP-CURL
- "HTTP CURL"
- "This enables RESTProtocol, InvokeHTTP, and the
HTTPClient for Site to Site"
- "extensions/http-curl"
- "${TEST_DIR}/curl-tests")
+createExtension(DISABLE_CURL
+HTTP-CURL
--- End diff --
I don't follow. What do you need me to change?
---
[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r147534705
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,309 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+
+#include
+#include
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to
perform on the archive", "");
+core::Property ManipulateArchive::Target("Target", "The path within the
archive to perform the operation on", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination
for operations (move or copy) which result in new entries", "");
+core::Property ManipulateArchive::Before("Before", "For operations which
result in new entries, places the new entry before the entry specified by this
property", "");
+core::Property ManipulateArchive::After("After", "For operations which
result in new entries, places the new entry after the entry specified by this
property", "");
+core::Relationship ManipulateArchive::Success("success", "success
operational on the flow record");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY = "copy";
+char const* ManipulateArchive::OPERATION_MOVE = "move";
+char const* ManipulateArchive::OPERATION_TOUCH = "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onTrigger(core::ProcessContext *context,
core::ProcessSession *session) {
+std::shared_ptr flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+std::string operation;
+context->getProperty(Operation.getName(), operation);
+
+std::string targetEntry;
+context->getProperty(Target.getName(), targetEntry);
+
+std::string destination;
+context->getProperty(Destination.getName(), destination);
+
+std::string before;
+context->getProperty(Before.getName(), before);
+
+std::string after;
+context->getProperty(After.getName(), after);
+
+// TODO(calebj) Validate properties
+
+FocusArchiveEntry::ArchiveMetadata archiveMetadata;
+FocusArchiveEntry::ReadCallback readCallback(this, &archiveMetadata);
+session->read(flowFile, &readCallback);
+
+logger_->log_info("ManipulateArchive performing operation %s on %s",
operation.c_str(), targetEntry.c_str());
+
+// Perform operation: REMOVE
+if (operation == OPERATION_REMOVE) {
+for (auto it = archiveMetadata.entryMetadata.begin(); it !=
archiveMetadata.entryMetadata.end();) {
+if ((*it).entryName == targetEntry) {
+logger_->log_info("ManipulateArchive found entry %s for
removal", targetEntry.c_str());
+std::remove((*it).tmpFileName.c_str());
--- End diff --
I'm leaning towards yes. There's no surefire way to determine the
uncompressed size of input files.
---
[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/156#discussion_r147534442
--- Diff: extensions/libarchive/ManipulateArchive.cpp ---
@@ -0,0 +1,309 @@
+/**
+ * @file ManipulateArchive.cpp
+ * ManipulateArchive class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+
+#include
+#include
+
+#include "ManipulateArchive.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ManipulateArchive::Operation("Operation", "Operation to
perform on the archive", "");
+core::Property ManipulateArchive::Target("Target", "The path within the
archive to perform the operation on", "");
+core::Property ManipulateArchive::Destination("Destination", "Destination
for operations (move or copy) which result in new entries", "");
+core::Property ManipulateArchive::Before("Before", "For operations which
result in new entries, places the new entry before the entry specified by this
property", "");
+core::Property ManipulateArchive::After("After", "For operations which
result in new entries, places the new entry after the entry specified by this
property", "");
+core::Relationship ManipulateArchive::Success("success", "success
operational on the flow record");
+
+char const* ManipulateArchive::OPERATION_REMOVE = "remove";
+char const* ManipulateArchive::OPERATION_COPY = "copy";
+char const* ManipulateArchive::OPERATION_MOVE = "move";
+char const* ManipulateArchive::OPERATION_TOUCH = "touch";
+
+void ManipulateArchive::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Operation);
+properties.insert(Target);
+properties.insert(Destination);
+properties.insert(Before);
+properties.insert(After);
+setSupportedProperties(properties);
+
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ManipulateArchive::onTrigger(core::ProcessContext *context,
core::ProcessSession *session) {
+std::shared_ptr flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+std::string operation;
+context->getProperty(Operation.getName(), operation);
+
+std::string targetEntry;
+context->getProperty(Target.getName(), targetEntry);
+
+std::string destination;
+context->getProperty(Destination.getName(), destination);
+
+std::string before;
+context->getProperty(Before.getName(), before);
+
+std::string after;
+context->getProperty(After.getName(), after);
+
+// TODO(calebj) Validate properties
+
+FocusArchiveEntry::ArchiveMetadata archiveMetadata;
+FocusArchiveEntry::ReadCallback readCallback(this, &archiveMetadata);
+session->read(flowFile, &readCallback);
+
+logger_->log_info("ManipulateArchive performing operation %s on %s",
operation.c_str(), targetEntry.c_str());
+
+// Perform operation: REMOVE
+if (operation == OPERATION_REMOVE) {
+for (auto it = archiveMetadata.entryMetadata.begin(); it !=
archiveMetadata.entryMetadata.end();) {
+if ((*it).entryName == targetEntry) {
+logger_->log_info("ManipulateArchive found entry %s for
r
[GitHub] nifi-minifi-cpp pull request #157: MINIFI-269 Implement ApplyTemplate proces...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/157 MINIFI-269 Implement ApplyTemplate processor Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp ApplyTemplate Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/157.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #157 commit 2913bc1d456adfeb1c6ac58ea91c061b4451e02d Author: Caleb Johnson Date: 2017-10-17T19:13:32Z MINIFI-269 Implement ApplyTemplate processor ---
[GitHub] nifi-minifi-cpp pull request #156: MINIFICPP-268 Implement ManipulateArchive...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/156 MINIFICPP-268 Implement ManipulateArchive processor Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp ManipulateArchive Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/156.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #156 ---
[GitHub] nifi-minifi-cpp issue #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/148 @phrocker That was the last one, I promise. ---
[GitHub] nifi-minifi-cpp pull request #:
Github user calebj commented on the pull request: https://github.com/apache/nifi-minifi-cpp/commit/a0268cf374c3e632fde89352d77dcf1be29d75dc#commitcomment-25237614 In .travis.yml: In .travis.yml on line 68: It was the only way to get the build under Travis's 50 minute limit, heh. Some tests on OSX randomly fail sometimes. I don't have a mac to test on here, unfortunately. I'll need help isolating it if it becomes a problem. I think sharing some of the objects like TestBase.cpp.o between tests would help with speeding this up, too. If it's just a matter of config changes, implementing parallel ctest should be a snap. ---
[GitHub] nifi-minifi-cpp issue #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/148 The build fails both with or without the `struct` keyword; I'll try a different approach next. There are also some other things I forgot to add when moving to the extension. ---
[GitHub] nifi-minifi-cpp issue #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/148 @phrocker I already have a travis-ci.org account, but I've granted access to @NifiLocal and synced the account. ---
[GitHub] nifi-minifi-cpp pull request #152: MINIFICPP-52 basic ExtractText processor
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/152#discussion_r146919029
--- Diff: libminifi/include/processors/ExtractText.h ---
@@ -0,0 +1,91 @@
+/**
+ * @file ExtractText.h
+ * ExtractText class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef __EXTRACT_TEXT_H__
+#define __EXTRACT_TEXT_H__
+
+#include "FlowFileRecord.h"
+#include "core/Processor.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+//! ExtractText Class
+class ExtractText : public core::Processor {
+public:
+//! Constructor
+/*!
+ * Create a new processor
+ */
+ExtractText(std::string name, uuid_t uuid = NULL)
+: Processor(name, uuid)
+{
+logger_ = logging::LoggerFactory::getLogger();
+}
+//! Destructor
+virtual ~ExtractText()
+{
+}
+//! Processor Name
+static constexpr char const* ProcessorName = "MergeContent";
+//! Supported Properties
+static core::Property Attribute;
+//! Supported Relationships
+static core::Relationship Success;
+
+//! OnTrigger method, implemented by NiFi ExtractText
+virtual void onTrigger(core::ProcessContext *context,
core::ProcessSession *session);
+//! Initialize, over write by NiFi ExtractText
+virtual void initialize(void);
+
+class ReadCallback : public InputStreamCallback {
+public:
+ReadCallback(std::shared_ptr flowFile,
core::ProcessContext *ct);
+~ReadCallback() { delete[] _buffer; }
+int64_t process(std::shared_ptr stream);
+
+private:
+std::shared_ptr logger_;
+std::shared_ptr _flowFile;
+core::ProcessContext *_ctx;
+uint8_t *_buffer;
--- End diff --
I was finally able to get it working. readData doesn't complain, but I
think I'll stick to calling `write()` on the block directly. The alternative is
to use
```
std::copy(buffer_.begin(), buffer_.end(),
std::ostream_iterator(contentStream));
```
which checks each character individually before copying, instead of doing
it as one block.
---
[GitHub] nifi-minifi-cpp pull request #152: MINIFICPP-52 basic ExtractText processor
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/152#discussion_r146907747
--- Diff: libminifi/include/processors/ExtractText.h ---
@@ -0,0 +1,91 @@
+/**
+ * @file ExtractText.h
+ * ExtractText class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef __EXTRACT_TEXT_H__
+#define __EXTRACT_TEXT_H__
+
+#include "FlowFileRecord.h"
+#include "core/Processor.h"
+#include "core/ProcessSession.h"
+#include "core/Resource.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+//! ExtractText Class
+class ExtractText : public core::Processor {
+public:
+//! Constructor
+/*!
+ * Create a new processor
+ */
+ExtractText(std::string name, uuid_t uuid = NULL)
+: Processor(name, uuid)
+{
+logger_ = logging::LoggerFactory::getLogger();
+}
+//! Destructor
+virtual ~ExtractText()
+{
+}
+//! Processor Name
+static constexpr char const* ProcessorName = "MergeContent";
+//! Supported Properties
+static core::Property Attribute;
+//! Supported Relationships
+static core::Relationship Success;
+
+//! OnTrigger method, implemented by NiFi ExtractText
+virtual void onTrigger(core::ProcessContext *context,
core::ProcessSession *session);
+//! Initialize, over write by NiFi ExtractText
+virtual void initialize(void);
+
+class ReadCallback : public InputStreamCallback {
+public:
+ReadCallback(std::shared_ptr flowFile,
core::ProcessContext *ct);
+~ReadCallback() { delete[] _buffer; }
+int64_t process(std::shared_ptr stream);
+
+private:
+std::shared_ptr logger_;
+std::shared_ptr _flowFile;
+core::ProcessContext *_ctx;
+uint8_t *_buffer;
--- End diff --
I wasn't able to get `stream->readData(buf_vec)` working. It appears to
fill the vector with null bytes. I'll take care of the other things and try it
again from scratch.
---
[GitHub] nifi-minifi-cpp issue #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/148 @phrocker Cool. I can squash into one commit and force push, if needed. ---
[GitHub] nifi-minifi-cpp pull request #152: MINIFICPP-52 basic ExtractText processor
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/152#discussion_r146861542
--- Diff: libminifi/src/processors/ExtractText.cpp ---
@@ -0,0 +1,105 @@
+/**
+ * @file ExtractText.cpp
+ * ExtractText class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+
+#include "processors/ExtractText.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ExtractText::Attribute("Attribute", "Attribute to set from
content (TEMPORARY)", "");
+core::Relationship ExtractText::Success("success", "success operational on
the flow record");
+
+void ExtractText::initialize() {
+//! Set the supported properties
+std::set properties;
+properties.insert(Attribute);
+setSupportedProperties(properties);
+//! Set the supported relationships
+std::set relationships;
+relationships.insert(Success);
+setSupportedRelationships(relationships);
+}
+
+void ExtractText::onTrigger(core::ProcessContext *context,
core::ProcessSession *session) {
+std::shared_ptr flowFile = session->get();
+
+if (!flowFile) {
+return;
+}
+
+ReadCallback cb(flowFile, context);
+session->read(flowFile, &cb);
+session->transfer(flowFile, Success);
+}
+
+int64_t ExtractText::ReadCallback::process(std::shared_ptr
stream) {
+int64_t ret = 0;
+uint64_t read_size = 0;
+
+std::string attrKey;
+_ctx->getProperty(Attribute.getName(), attrKey);
+std::stringstream contentStream(std::stringstream::out |
std::stringstream::in);
+std::string contentStr;
+
+while (read_size < _flowFile->getSize()) {
+ret = stream->read(_buffer, _max_read);
+if (ret < 0) {
+return -1;
+}
+
+if (ret > 0) {
+contentStream.write(reinterpret_cast(_buffer),
ret);
--- End diff --
Sure. Should it be a constant somewhere, or a property?
---
[GitHub] nifi-minifi-cpp pull request #152: MINIFICPP-52 basic ExtractText processor
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/152#discussion_r146861365
--- Diff: libminifi/src/processors/ExtractText.cpp ---
@@ -0,0 +1,105 @@
+/**
+ * @file ExtractText.cpp
+ * ExtractText class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+#include
+#include
+
+#include "processors/ExtractText.h"
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+#include "core/FlowFile.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property ExtractText::Attribute("Attribute", "Attribute to set from
content (TEMPORARY)", "");
--- End diff --
Not sure, ask @achristianson. I got it from
[here](https://github.com/NiFiLocal/nifi-minifi-cpp/commit/ec03fd15f80e33f1dba3be9c1e2520745e732930#diff-3d4ecb8ad9604bc01f1c18e283d8bc4fR31).
---
[GitHub] nifi-minifi-cpp issue #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/148 I'm changing the types for size, mtime and mtime_nsec in `ArchiveEntryMetadata`'s declaration instead of casting. Why'd you take out the boost header include? ---
[GitHub] nifi-minifi-cpp pull request #152: MINIFICPP-52 basic ExtractText processor
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/152 MINIFICPP-52 basic ExtractText processor Regex support blocked by dynamic properties, #37 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp ExtractText Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/152.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #152 ---
[GitHub] nifi-minifi-cpp issue #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/148 @phrocker For the first error, I'm pretty sure the only type it can convert to is int64. The JsonCPP docs don't have any ambiguities there, it's just signed and unsigned int and int64. For the second, I don't see how it's even possible since archive.h is included at the top of that file. Can you check to see if it's using the thirdparty/ headers, or could it be pulling them from somewhere else on your system? ---
[GitHub] nifi-minifi-cpp issue #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/148 Thank @achristianson for the original implementation, way back in March/April against 070d8758fded4836f17816c1b1a34a48aa1bba20. I just brought it up to date. ---
[GitHub] nifi-minifi-cpp issue #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/148 Rebased. ---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145747588 --- Diff: libminifi/include/core/FlowConfiguration.h --- @@ -35,6 +35,8 @@ #include "processors/ExecuteProcess.h" #include "processors/AppendHostInfo.h" #include "processors/MergeContent.h" +#include "processors/FocusArchiveEntry.h" --- End diff -- Clarified in the latest commit ---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145747665
--- Diff: libminifi/include/FlowFileRecord.h ---
@@ -164,6 +164,11 @@ class FlowFileRecord : public core::FlowFile, public
io::Serializable {
return content_full_fath_;
}
+ /**
+ * Cleanly relinquish a resource claim
+ */
--- End diff --
Clarified in the latest commit
---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145743863
--- Diff: libminifi/src/processors/FocusArchiveEntry.cpp ---
@@ -0,0 +1,340 @@
+/**
+ * @file FocusArchiveEntry.cpp
+ * FocusArchiveEntry class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "processors/FocusArchiveEntry.h"
+
+#include
+#include
+
+#include
+
+#include
+
+#include
+#include
+
+#include
+#include
+#include
+
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+
+#include "json/json.h"
+#include "json/writer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property FocusArchiveEntry::Path(
+"Path",
+"The path within the archive to focus (\"/\" to focus the total
archive)",
+"");
+core::Relationship FocusArchiveEntry::Success(
+"success",
+"success operational on the flow record");
+
+bool
FocusArchiveEntry::set_del_or_update_attr(std::shared_ptr
flowFile, const std::string key, std::string* value) const {
+ if (value == nullptr)
+return flowFile->removeAttribute(key);
+ else if (flowFile->updateAttribute(key, *value))
+return true;
+ else
+return flowFile->addAttribute(key, *value);
+}
+
+void FocusArchiveEntry::initialize() {
+ //! Set the supported properties
+ std::set properties;
+ properties.insert(Path);
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(Success);
+ setSupportedRelationships(relationships);
+}
+
+void FocusArchiveEntry::onTrigger(core::ProcessContext *context,
+ core::ProcessSession *session) {
+ auto flowFile = session->get();
+ std::shared_ptr flowFileRecord =
std::static_pointer_cast(flowFile);
+
+ if (!flowFile) {
+return;
+ }
+
+ std::string targetEntry;
+ context->getProperty(Path.getName(), targetEntry);
+
+ // Extract archive contents
+ ArchiveMetadata archiveMetadata;
+ archiveMetadata.focusedEntry = targetEntry;
+ ReadCallback cb(&archiveMetadata);
+ session->read(flowFile, &cb);
+
+ // For each extracted entry, import & stash to key
+ std::string targetEntryStashKey;
+
+ for (auto &entryMetadata : archiveMetadata.entryMetadata) {
+if (entryMetadata.entryType == AE_IFREG) {
+ logger_->log_info("FocusArchiveEntry importing %s from %s",
+ entryMetadata.entryName.c_str(),
+ entryMetadata.tmpFileName.c_str());
+ session->import(entryMetadata.tmpFileName, flowFile, false, 0);
+ char stashKey[37];
+ uuid_t stashKeyUuid;
+ uuid_generate(stashKeyUuid);
+ uuid_unparse_lower(stashKeyUuid, stashKey);
+ logger_->log_debug(
+ "FocusArchiveEntry generated stash key %s for entry %s",
+ stashKey,
+ entryMetadata.entryName.c_str());
+ entryMetadata.stashKey.assign(stashKey);
+
+ if (entryMetadata.entryName == targetEntry) {
+targetEntryStashKey = entryMetadata.stashKey;
+ }
+
+ // Stash the content
+ session->stash(entryMetadata.stashKey, flowFile);
+}
+ }
+
+ // Restore target archive entry
+ if (targetEntryStashKey != "") {
+session->restore(targetEntryStashKey, flowFile);
+ } else {
+logger_->log_warn(
+ "FocusArchiveEntry failed to locate target entry: %s",
+ targetEntry.c_str());
+ }
+
+
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145744191
--- Diff: libminifi/include/processors/FocusArchiveEntry.h ---
@@ -0,0 +1,115 @@
+/**
+ * @file FocusArchiveEntry.h
+ * FocusArchiveEntry class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef LIBMINIFI_INCLUDE_PROCESSORS_FOCUSARCHIVEENTRY_H_
+#define LIBMINIFI_INCLUDE_PROCESSORS_FOCUSARCHIVEENTRY_H_
+
+#include
+#include
+#include
+
+#include "FlowFileRecord.h"
+#include "core/Processor.h"
+#include "core/ProcessSession.h"
+#include "core/Core.h"
+#include "core/logging/LoggerConfiguration.h"
+#include "core/Resource.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+using logging::LoggerFactory;
+
+//! FocusArchiveEntry Class
+class FocusArchiveEntry : public core::Processor {
+ public:
+ //! Constructor
+ /*!
+ * Create a new processor
+ */
+ explicit FocusArchiveEntry(std::string name, uuid_t uuid = NULL)
+ : core::Processor(name, uuid),
+logger_(logging::LoggerFactory::getLogger()) {
+ }
+ //! Destructor
+ virtual ~FocusArchiveEntry() {
+ }
+ //! Processor Name
+ static constexpr char const* ProcessorName = "FocusArchiveEntry";
+ //! Supported Properties
+ static core::Property Path;
+ //! Supported Relationships
+ static core::Relationship Success;
+
+ bool set_del_or_update_attr(std::shared_ptr, const
std::string, std::string*) const;
--- End diff --
Fixed in d2e7e34ab8b331ac484b9b16bd51455799a1502b
---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145743845 --- Diff: LICENSE --- @@ -534,4 +534,68 @@ ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +This projects includes libarchive bundle (https://www.libarchive.org) +which is available under a BSD License by Tim Kientzle and others + --- End diff -- Fixed in d2e7e34ab8b331ac484b9b16bd51455799a1502b ---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145744366 --- Diff: libminifi/include/core/ProcessSession.h --- @@ -19,6 +19,7 @@ #define __PROCESS_SESSION_H__ #include +#include --- End diff -- Fixed in d2e7e34ab8b331ac484b9b16bd51455799a1502b ---
[GitHub] nifi-minifi-cpp issue #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/148 Things left to look into: * [ ] Move things which depend on libarchive to an extension * [ ] Use streams for exportContent * [ ] Move exportContent ReadCallback to another file ---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145745217
--- Diff: libminifi/include/processors/FocusArchiveEntry.h ---
@@ -0,0 +1,115 @@
+/**
+ * @file FocusArchiveEntry.h
+ * FocusArchiveEntry class declaration
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef LIBMINIFI_INCLUDE_PROCESSORS_FOCUSARCHIVEENTRY_H_
+#define LIBMINIFI_INCLUDE_PROCESSORS_FOCUSARCHIVEENTRY_H_
+
+#include
+#include
+#include
+
+#include "FlowFileRecord.h"
+#include "core/Processor.h"
+#include "core/ProcessSession.h"
+#include "core/Core.h"
+#include "core/logging/LoggerConfiguration.h"
+#include "core/Resource.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+using logging::LoggerFactory;
--- End diff --
Fixed in d2e7e34ab8b331ac484b9b16bd51455799a1502b
---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145744343
--- Diff: libminifi/include/core/FlowFile.h ---
@@ -50,6 +50,32 @@ class FlowFile : public core::Connectable {
void clearResourceClaim();
/**
+ * Returns a pointer to this flow file record's
+ * claim at the given stash key
+ */
+ std::shared_ptr getStashClaim(const std::string &key);
+
+ /**
+ * Sets the given stash key to the inbound claim argument
+ */
+ void setStashClaim(const std::string &key,
std::shared_ptr &claim);
--- End diff --
Fixed in d2e7e34ab8b331ac484b9b16bd51455799a1502b
---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145743879
--- Diff: libminifi/src/processors/FocusArchiveEntry.cpp ---
@@ -0,0 +1,340 @@
+/**
+ * @file FocusArchiveEntry.cpp
+ * FocusArchiveEntry class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "processors/FocusArchiveEntry.h"
+
+#include
+#include
+
+#include
+
+#include
+
+#include
+#include
+
+#include
+#include
+#include
+
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+
+#include "json/json.h"
+#include "json/writer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property FocusArchiveEntry::Path(
+"Path",
+"The path within the archive to focus (\"/\" to focus the total
archive)",
+"");
+core::Relationship FocusArchiveEntry::Success(
+"success",
+"success operational on the flow record");
+
+bool
FocusArchiveEntry::set_del_or_update_attr(std::shared_ptr
flowFile, const std::string key, std::string* value) const {
+ if (value == nullptr)
+return flowFile->removeAttribute(key);
+ else if (flowFile->updateAttribute(key, *value))
+return true;
+ else
+return flowFile->addAttribute(key, *value);
+}
+
+void FocusArchiveEntry::initialize() {
+ //! Set the supported properties
+ std::set properties;
+ properties.insert(Path);
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(Success);
+ setSupportedRelationships(relationships);
+}
+
+void FocusArchiveEntry::onTrigger(core::ProcessContext *context,
+ core::ProcessSession *session) {
+ auto flowFile = session->get();
+ std::shared_ptr flowFileRecord =
std::static_pointer_cast(flowFile);
+
+ if (!flowFile) {
+return;
+ }
+
+ std::string targetEntry;
+ context->getProperty(Path.getName(), targetEntry);
+
+ // Extract archive contents
+ ArchiveMetadata archiveMetadata;
+ archiveMetadata.focusedEntry = targetEntry;
+ ReadCallback cb(&archiveMetadata);
+ session->read(flowFile, &cb);
+
+ // For each extracted entry, import & stash to key
+ std::string targetEntryStashKey;
+
+ for (auto &entryMetadata : archiveMetadata.entryMetadata) {
+if (entryMetadata.entryType == AE_IFREG) {
+ logger_->log_info("FocusArchiveEntry importing %s from %s",
+ entryMetadata.entryName.c_str(),
+ entryMetadata.tmpFileName.c_str());
+ session->import(entryMetadata.tmpFileName, flowFile, false, 0);
+ char stashKey[37];
+ uuid_t stashKeyUuid;
+ uuid_generate(stashKeyUuid);
+ uuid_unparse_lower(stashKeyUuid, stashKey);
+ logger_->log_debug(
+ "FocusArchiveEntry generated stash key %s for entry %s",
+ stashKey,
+ entryMetadata.entryName.c_str());
+ entryMetadata.stashKey.assign(stashKey);
+
+ if (entryMetadata.entryName == targetEntry) {
+targetEntryStashKey = entryMetadata.stashKey;
+ }
+
+ // Stash the content
+ session->stash(entryMetadata.stashKey, flowFile);
+}
+ }
+
+ // Restore target archive entry
+ if (targetEntryStashKey != "") {
+session->restore(targetEntryStashKey, flowFile);
+ } else {
+logger_->log_warn(
+ "FocusArchiveEntry failed to locate target entry: %s",
+ targetEntry.c_str());
+ }
+
+
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request:
https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145743902
--- Diff: libminifi/src/processors/FocusArchiveEntry.cpp ---
@@ -0,0 +1,340 @@
+/**
+ * @file FocusArchiveEntry.cpp
+ * FocusArchiveEntry class implementation
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "processors/FocusArchiveEntry.h"
+
+#include
+#include
+
+#include
+
+#include
+
+#include
+#include
+
+#include
+#include
+#include
+
+#include "core/ProcessContext.h"
+#include "core/ProcessSession.h"
+
+#include "json/json.h"
+#include "json/writer.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property FocusArchiveEntry::Path(
+"Path",
+"The path within the archive to focus (\"/\" to focus the total
archive)",
+"");
+core::Relationship FocusArchiveEntry::Success(
+"success",
+"success operational on the flow record");
+
+bool
FocusArchiveEntry::set_del_or_update_attr(std::shared_ptr
flowFile, const std::string key, std::string* value) const {
+ if (value == nullptr)
+return flowFile->removeAttribute(key);
+ else if (flowFile->updateAttribute(key, *value))
+return true;
+ else
+return flowFile->addAttribute(key, *value);
+}
+
+void FocusArchiveEntry::initialize() {
+ //! Set the supported properties
+ std::set properties;
+ properties.insert(Path);
+ setSupportedProperties(properties);
+ //! Set the supported relationships
+ std::set relationships;
+ relationships.insert(Success);
+ setSupportedRelationships(relationships);
+}
+
+void FocusArchiveEntry::onTrigger(core::ProcessContext *context,
+ core::ProcessSession *session) {
+ auto flowFile = session->get();
+ std::shared_ptr flowFileRecord =
std::static_pointer_cast(flowFile);
+
+ if (!flowFile) {
+return;
+ }
+
+ std::string targetEntry;
+ context->getProperty(Path.getName(), targetEntry);
+
+ // Extract archive contents
+ ArchiveMetadata archiveMetadata;
+ archiveMetadata.focusedEntry = targetEntry;
+ ReadCallback cb(&archiveMetadata);
+ session->read(flowFile, &cb);
+
+ // For each extracted entry, import & stash to key
+ std::string targetEntryStashKey;
+
+ for (auto &entryMetadata : archiveMetadata.entryMetadata) {
+if (entryMetadata.entryType == AE_IFREG) {
+ logger_->log_info("FocusArchiveEntry importing %s from %s",
+ entryMetadata.entryName.c_str(),
+ entryMetadata.tmpFileName.c_str());
+ session->import(entryMetadata.tmpFileName, flowFile, false, 0);
+ char stashKey[37];
+ uuid_t stashKeyUuid;
--- End diff --
Fixed in d2e7e34ab8b331ac484b9b16bd51455799a1502b
---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145742734 --- Diff: LICENSE --- @@ -534,4 +534,68 @@ ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +This projects includes libarchive bundle (https://www.libarchive.org) +which is available under a BSD License by Tim Kientzle and others + +Copyright (c) 2003-2009 Tim Kientzle and other authors +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer + in this position and unchanged. +2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR +IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES +OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. +IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, +INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT +NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF +THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (END LICENSE TEXT) + +All libarchive C sources (including .c and .h files) +and documentation files are subject to the copyright notice reproduced +above. + +This libarchive includes below files +libarchive/archive_entry.c +libarchive/archive_read_support_filter_compress.c +libarchive/archive_write_add_filter_compress.c +which under a 3-clause UC Regents copyright as below +/*- + * Copyright (c) 1993 + * The Regents of the University of California. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + *notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + *notice, this list of conditions and the following disclaimer in the + *documentation and/or other materials provided with the distribution. + * 4. Neither the name of the University nor the names of its contributors --- End diff -- Neither. It's the 3-clause UC Regents copyright, which is in the three aforementioned source files. The originals also skip the third list item. ---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
Github user calebj commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/148#discussion_r145738276 --- Diff: CMakeLists.txt --- @@ -101,6 +101,7 @@ set(CIVETWEB_ENABLE_SSL_DYNAMIC_LOADING OFF CACHE BOOL "Disable dynamic SSL libr set(CIVETWEB_ENABLE_CXX ON CACHE BOOL "Enable civet C++ library") add_subdirectory(thirdparty/yaml-cpp-yaml-cpp-0.5.3) add_subdirectory(thirdparty/civetweb-1.9.1 EXCLUDE_FROM_ALL) +add_subdirectory(thirdparty/libarchive-3.3.2) --- End diff -- I'm not sure how to do that, are there any examples? ---
[GitHub] nifi-minifi-cpp pull request #148: MINIFI-244 Un/FocusArchive processors
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/148 MINIFI-244 Un/FocusArchive processors Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. Split into two commits to share a common base with #146. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp MINIFI-244-rc2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/148.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #148 commit 5cf2ae943e9eb2ab8c289fc25da451b96da38a04 Author: Caleb Johnson Date: 2017-10-18T18:21:29Z Pull in MINIFICPP-72's libarchive commit 3a32355c8f81e7216fca5ba8a83e9781aaeedb04 Author: Caleb Johnson Date: 2017-10-17T15:45:53Z MINIFI-244 Un/FocusArchive processors ---
[GitHub] nifi-minifi-cpp pull request #139: MINIFI-388 Backport the rest of EVP_Diges...
GitHub user calebj opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/139 MINIFI-388 Backport the rest of EVP_Digest for civetweb As per the discussion in apache/nifi-minifi-cpp#131. This is a backport of the `#define` declaration of `EVP_Digest()` and the matching entry in `crypto_sw` from civetweb/civetweb@fb9ef36. Without this patch, minifi fails to build on Ubuntu 14.04 with GCC 4.8.4, against OpenSSL version is 1.0.1f-1ubuntu2.22, with: ``` [ 45%] Building C object thirdparty/civetweb-1.9.1/src/CMakeFiles/c-library.dir/civetweb.c.o /home/ubuntu/workspace/thirdparty/civetweb-1.9.1/src/civetweb.c: In function âssl_get_client_cert_infoâ: /home/ubuntu/workspace/thirdparty/civetweb-1.9.1/src/civetweb.c:11873:4: error: implicit declaration of function âEVP_Digestâ [-Wimplicit-function-declaration] if (!EVP_Digest( ^ make[2]: *** [thirdparty/civetweb-1.9.1/src/CMakeFiles/c-library.dir/civetweb.c.o] Error 1 ``` Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/NiFiLocal/nifi-minifi-cpp civet_evp_digest_backport Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/139.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #139 commit 66dc8f039ca0fa1af362c11a5bcc43475456bc74 Author: Caleb Johnson Date: 2017-09-28T11:25:18Z MINIFI-388 Backport the rest of EVP_Digest for civetweb ---
[GitHub] nifi-minifi-cpp issue #131: MINIFI-388 symlink versioned libcrypto/libssl to...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/131 I'm building on Cloud9, which has Ubuntu 14.04 with GCC 4.8.4, against OpenSSL version is 1.0.1f-1ubuntu2.22 if it helps. The specific error was: ``` [ 45%] Building C object thirdparty/civetweb-1.9.1/src/CMakeFiles/c-library.dir/civetweb.c.o /home/ubuntu/workspace/thirdparty/civetweb-1.9.1/src/civetweb.c: In function âssl_get_client_cert_infoâ: /home/ubuntu/workspace/thirdparty/civetweb-1.9.1/src/civetweb.c:11873:4: error: implicit declaration of function âEVP_Digestâ [-Wimplicit-function-declaration] if (!EVP_Digest( ^ make[2]: *** [thirdparty/civetweb-1.9.1/src/CMakeFiles/c-library.dir/civetweb.c.o] Error 1 ``` ---
[GitHub] nifi-minifi-cpp issue #131: MINIFI-388 symlink versioned libcrypto/libssl to...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/131 This doesn't build for me without backporting the `#define` for EVP_Digest and the matching entry in `crypto_sw`. I added these in 66dc8f039ca0fa1af362c11a5bcc43475456bc74; should I create another PR? ---
