nifi-site git commit: Restored warning about automated downloads from archive.apache.org on downloads page.
Repository: nifi-site Updated Branches: refs/heads/master 14e0e8660 -> 52b170bb0 Restored warning about automated downloads from archive.apache.org on downloads page. Project: http://git-wip-us.apache.org/repos/asf/nifi-site/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi-site/commit/52b170bb Tree: http://git-wip-us.apache.org/repos/asf/nifi-site/tree/52b170bb Diff: http://git-wip-us.apache.org/repos/asf/nifi-site/diff/52b170bb Branch: refs/heads/master Commit: 52b170bb0b4f14e4a8cf6b02b6ff53e66c60c974 Parents: 14e0e86 Author: Andy LoPresto Authored: Tue Nov 27 21:10:07 2018 -0800 Committer: Andy LoPresto Committed: Tue Nov 27 21:10:07 2018 -0800 -- src/pages/html/download.hbs | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/nifi-site/blob/52b170bb/src/pages/html/download.hbs -- diff --git a/src/pages/html/download.hbs b/src/pages/html/download.hbs index 5617dd4..f3f5e72 100644 --- a/src/pages/html/download.hbs +++ b/src/pages/html/download.hbs @@ -51,6 +51,7 @@ title: Apache NiFi Downloads 1.7.1 Released July 16, 2018 + Note: The download links pull from archive.apache.org. Apache limits how much can be downloaded from this site per day so please avoid automated/continuous downloads from the archives Sources:
svn commit: r1847616 - /nifi/site/trunk/download.html
Author: alopresto Date: Wed Nov 28 05:12:24 2018 New Revision: 1847616 URL: http://svn.apache.org/viewvc?rev=1847616&view=rev Log: Restored warning about automated downloads from archive.apache.org on downloads page. Modified: nifi/site/trunk/download.html Modified: nifi/site/trunk/download.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/download.html?rev=1847616&r1=1847615&r2=1847616&view=diff == --- nifi/site/trunk/download.html (original) +++ nifi/site/trunk/download.html Wed Nov 28 05:12:24 2018 @@ -155,6 +155,7 @@ 1.7.1 Released July 16, 2018 + Note: The download links pull from archive.apache.org. Apache limits how much can be downloaded from this site per day so please avoid automated/continuous downloads from the archives Sources:
nifi-site git commit: Added instructions on maintaining existing versions of component documentation to Release Guide.
Repository: nifi-site
Updated Branches:
refs/heads/master bdeef60f2 -> 14e0e8660
Added instructions on maintaining existing versions of component documentation
to Release Guide.
Project: http://git-wip-us.apache.org/repos/asf/nifi-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi-site/commit/14e0e866
Tree: http://git-wip-us.apache.org/repos/asf/nifi-site/tree/14e0e866
Diff: http://git-wip-us.apache.org/repos/asf/nifi-site/diff/14e0e866
Branch: refs/heads/master
Commit: 14e0e86601c39f1c67c379e04d374a6192d6c3ad
Parents: bdeef60
Author: Andy LoPresto
Authored: Tue Nov 27 21:00:03 2018 -0800
Committer: Andy LoPresto
Committed: Tue Nov 27 21:00:03 2018 -0800
--
src/pages/markdown/release-guide.md | 14 +++---
1 file changed, 11 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi-site/blob/14e0e866/src/pages/markdown/release-guide.md
--
diff --git a/src/pages/markdown/release-guide.md
b/src/pages/markdown/release-guide.md
index acef254..46ef9ba 100644
--- a/src/pages/markdown/release-guide.md
+++ b/src/pages/markdown/release-guide.md
@@ -468,11 +468,19 @@ in the archive location so no need to do anything else.
1. Update the [Migration Guide][nifi-migration-guide] on the Wiki.
-1. If the release is on the latest development line, update the NiFi website
documentation pages to match the release.
+1. If the release is on the latest development line, update the NiFi website
documentation pages to match the release. _(This section will be updated when a
complete plan for maintaining different versions of the guides, API docs, etc.
is available.)_
1. Run the NiFi ${NIFI_VERSION}
1. Pull down the documentation by running `wget -prk
http://${host}:${port}/nifi-docs/documentation`
-1. Rename the file index file that was generated by running `mv
${host}:${port}/nifi-docs/documentation ${host}:${port}/nifi-docs/index.html`
-1. Use svn to replace the `nifi` and `nifi-docs` folders in
`https://svn.apache.org/repos/asf/nifi/site/trunk/docs` with those in the
`${host}:${port}` directory pulled down above
+1. Rename the directory to avoid escaping characters by running `mv -v
${host}\:${port} ${NIFI_VERSION}-docs`
+1. Rename the file index file that was generated by running `mv -v
${NIFI_VERSION}-docs/nifi-docs/documentation
${NIFI_VERSION}-docs/nifi-docs/index.html`
+1. Merge the existing documentation (i.e. version _n-1_, _n-2_, etc.) with
the new version documentation _(`rsync` is used to avoid issues with recursion
and merging)_
+1. Checkout the existing Subversion repository containing the docs by
running `svn co https://svn.apache.org/repos/asf/nifi/site/trunk/docs svn-docs`
+1. Replace the `nifi` directory (site JS assets, images, etc.) by
running `rsync -av ${NIFI_VERSION}-docs/nifi/ svn-docs/nifi/`
+1. Replace the `nifi-docs/...` directories (guides, API docs, JS/CSS
assets, images, etc. but _excluding_ `component` docs) by running `rsync -av
--delete --exclude='components' ${NIFI_VERSION}-docs/nifi-docs/
svn-docs/nifi-docs/` (the trailing slashes are important)
+1. Merge the component documentation by running `rsync -av
${NIFI_VERSION}/nifi-docs/components/ svn-docs/nifi-docs/components/`
+1. (Optional) Check the status by running `svn st svn-docs`
+1. Add the new files to version control by running `cd svn-docs && svn
add . --force`
+1. Commit the changes by running `svn ci -m "Added ${NIFI_VERSION}
docs to NiFi site."`
1. In JIRA mark the release version as 'Released' and 'Archived' through
'version' management in the 'administration' console.
svn commit: r1847615 - /nifi/site/trunk/release-guide.html
Author: alopresto
Date: Wed Nov 28 05:02:50 2018
New Revision: 1847615
URL: http://svn.apache.org/viewvc?rev=1847615&view=rev
Log:
Added instructions on maintaining existing version component documentation to
Release Guide.
Modified:
nifi/site/trunk/release-guide.html
Modified: nifi/site/trunk/release-guide.html
URL:
http://svn.apache.org/viewvc/nifi/site/trunk/release-guide.html?rev=1847615&r1=1847614&r2=1847615&view=diff
==
--- nifi/site/trunk/release-guide.html (original)
+++ nifi/site/trunk/release-guide.html Wed Nov 28 05:02:50 2018
@@ -545,12 +545,22 @@ the archive location. See current page
page as described delete artifacts other than the current/new release from the
dist/nifi SVN storage. They are already
in the archive location so no need to do anything else.
Update the https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance";>Migration
Guide on the Wiki.
-If the release is on the latest development line, update the NiFi
website documentation pages to match the release.
+If the release is on the latest development line, update the NiFi
website documentation pages to match the release. (This section will be
updated when a complete plan for maintaining different versions of the guides,
API docs, etc. is available.)
Run the NiFi ${NIFI_VERSION}
Pull down the documentation by running wget -prk
http://${host}:${port}/nifi-docs/documentation
-Rename the file index file that was generated by running mv
${host}:${port}/nifi-docs/documentation
${host}:${port}/nifi-docs/index.html
-Use svn to replace the nifi and nifi-docs
folders in https://svn.apache.org/repos/asf/nifi/site/trunk/docs
with those in the ${host}:${port} directory pulled down above
+Rename the directory to avoid escaping characters by running mv -v
${host}\:${port} ${NIFI_VERSION}-docs
+Rename the file index file that was generated by running mv -v
${NIFI_VERSION}-docs/nifi-docs/documentation
${NIFI_VERSION}-docs/nifi-docs/index.html
+Merge the existing documentation (i.e. version n-1, n-2,
etc.) with the new version documentation (rsync is used to
avoid issues with recursion and merging)
+
+Checkout the existing Subversion repository containing the docs by running
svn co https://svn.apache.org/repos/asf/nifi/site/trunk/docs
svn-docs
+Replace the nifi directory (site JS assets, images, etc.) by
running rsync -av ${NIFI_VERSION}-docs/nifi/ svn-docs/nifi/
+Replace the nifi-docs/... directories (guides, API docs,
JS/CSS assets, images, etc. but excluding component docs)
by running rsync -av --delete --exclude='components'
${NIFI_VERSION}-docs/nifi-docs/ svn-docs/nifi-docs/ (the trailing
slashes are important)
+Merge the component documentation by running rsync -av
${NIFI_VERSION}/nifi-docs/components/ svn-docs/nifi-docs/components/
+(Optional) Check the status by running svn st svn-docs
+Add the new files to version control by running cd svn-docs
&& svn add . --force
+Commit the changes by running svn ci -m "Added ${NIFI_VERSION}
docs to NiFi site."
+
In JIRA mark the release version as 'Released' and 'Archived' through
'version' management in the 'administration' console.
Ensure the release artifacts are successfully mirrored to the archive,
specifically
https://archive.apache.org/dist/nifi/${NIFI_VERSION}/nifi-${NIFI_VERSION}-bin.tar.gz.
svn commit: r1847605 - in /nifi/site/trunk/docs/nifi-docs/components/org.apache.nifi: nifi-ambari-nar/1.5.0/ nifi-ambari-nar/1.5.0/org.apache.nifi.reporting.ambari.AmbariReportingTask/ nifi-ambari-nar
Author: alopresto Date: Wed Nov 28 03:46:44 2018 New Revision: 1847605 URL: http://svn.apache.org/viewvc?rev=1847605&view=rev Log: Restored component docs for 1.5.0, 1.6.0, 1.7.0, and 1.7.1. [This commit notification would consist of 133 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]
nifi-minifi-cpp git commit: ExtractText processor doesn't handle "Size limit" property
Repository: nifi-minifi-cpp
Updated Branches:
refs/heads/master df2bb2549 -> e24d54e03
ExtractText processor doesn't handle "Size limit" property
This closes #446.
Signed-off-by: Marc Parisi
Project: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/commit/e24d54e0
Tree: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/tree/e24d54e0
Diff: http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/diff/e24d54e0
Branch: refs/heads/master
Commit: e24d54e03df86b53bd18aa6efb9ce2164ac9c003
Parents: df2bb25
Author: Arpad Boda
Authored: Tue Nov 20 13:56:48 2018 +0100
Committer: Marc Parisi
Committed: Tue Nov 27 08:54:08 2018 -0500
--
libminifi/include/processors/ExtractText.h | 5 ++--
libminifi/src/processors/ExtractText.cpp | 37 +++--
libminifi/test/unit/ExtractTextTests.cpp | 28 +--
3 files changed, 43 insertions(+), 27 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/e24d54e0/libminifi/include/processors/ExtractText.h
--
diff --git a/libminifi/include/processors/ExtractText.h
b/libminifi/include/processors/ExtractText.h
index b01665e..fcaf09d 100644
--- a/libminifi/include/processors/ExtractText.h
+++ b/libminifi/include/processors/ExtractText.h
@@ -56,9 +56,9 @@ public:
static constexpr int DEFAULT_SIZE_LIMIT = 2 * 1024 * 1024;
//! OnTrigger method, implemented by NiFi ExtractText
-void onTrigger(core::ProcessContext *context, core::ProcessSession
*session);
+void onTrigger(core::ProcessContext *context, core::ProcessSession
*session) override;
//! Initialize, over write by NiFi ExtractText
-void initialize(void);
+void initialize(void) override;
class ReadCallback : public InputStreamCallback {
public:
@@ -70,7 +70,6 @@ public:
std::shared_ptr flowFile_;
core::ProcessContext *ctx_;
std::vector buffer_;
-int64_t max_read_;
};
protected:
http://git-wip-us.apache.org/repos/asf/nifi-minifi-cpp/blob/e24d54e0/libminifi/src/processors/ExtractText.cpp
--
diff --git a/libminifi/src/processors/ExtractText.cpp
b/libminifi/src/processors/ExtractText.cpp
index 328a476..264838e 100644
--- a/libminifi/src/processors/ExtractText.cpp
+++ b/libminifi/src/processors/ExtractText.cpp
@@ -36,6 +36,8 @@ namespace nifi {
namespace minifi {
namespace processors {
+#define MAX_BUFFER_SIZE 4096
+
core::Property ExtractText::Attribute("Attribute", "Attribute to set from
content", "");
core::Property ExtractText::SizeLimit("Size Limit", "Maximum number of bytes
to read into the attribute. 0 for no limit. Default is 2MB.");
core::Relationship ExtractText::Success("success", "success operational on the
flow record");
@@ -44,6 +46,7 @@ void ExtractText::initialize() {
//! Set the supported properties
std::set properties;
properties.insert(Attribute);
+ properties.insert(SizeLimit);
setSupportedProperties(properties);
//! Set the supported relationships
std::set relationships;
@@ -65,9 +68,8 @@ void ExtractText::onTrigger(core::ProcessContext *context,
core::ProcessSession
int64_t ExtractText::ReadCallback::process(std::shared_ptr
stream) {
int64_t ret = 0;
- uint64_t size_limit = flowFile_->getSize();
uint64_t read_size = 0;
- uint64_t loop_read = max_read_;
+ uint64_t size_limit = flowFile_->getSize();
std::string attrKey, sizeLimitStr;
ctx_->getProperty(Attribute.getName(), attrKey);
@@ -79,39 +81,32 @@ int64_t
ExtractText::ReadCallback::process(std::shared_ptr strea
size_limit = std::stoi(sizeLimitStr);
std::ostringstream contentStream;
- std::string contentStr;
while (read_size < size_limit) {
-if (size_limit - read_size < (uint64_t) max_read_)
- loop_read = size_limit - read_size;
-
-ret = stream->readData(buffer_, loop_read);
-buffer_.resize(ret);
+// Don't read more than config limit or the size of the buffer
+ret = stream->readData(buffer_, std::min((size_limit -
read_size), buffer_.capacity()));
if (ret < 0) {
- return -1;
+ return -1; // Stream error
+} else if (ret == 0) {
+ break; // End of stream, no more data
}
-if (ret > 0) {
- contentStream.write(reinterpret_cast(&buffer_[0]), ret);
- if (contentStream.fail()) {
-return -1;
- }
-} else {
- break;
+contentStream.write(reinterpret_cast(&buffer_[0]), ret);
+read_size += ret;
+if (contentStream.fail()) {
+ return -1;
}
}
- contentStr = contentStream.str();
- flowFile_->setAttribute(attrKey, contentStr);
+ flowFile_->setAttribute(attrKey, contentStream.str());
return r
nifi git commit: NIFI-5834: Restore default PutHiveQL error handling behavior
Repository: nifi
Updated Branches:
refs/heads/master e603c486f -> 455e3c1bc
NIFI-5834: Restore default PutHiveQL error handling behavior
NIFI-5834: Incorporated review comments
This closes #3179.
Signed-off-by: Koji Kawamura
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/455e3c1b
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/455e3c1b
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/455e3c1b
Branch: refs/heads/master
Commit: 455e3c1bc8d4e12ea75d4e6ac2e4c58cbb535e5d
Parents: e603c48
Author: Matthew Burgess
Authored: Tue Nov 20 17:58:59 2018 -0500
Committer: Koji Kawamura
Committed: Tue Nov 27 18:10:26 2018 +0900
--
.../apache/nifi/processors/hive/PutHiveQL.java | 7 +-
.../nifi/processors/hive/TestPutHiveQL.java | 80 +++-
.../apache/nifi/processors/hive/PutHive3QL.java | 19 -
.../nifi/processors/hive/TestPutHive3QL.java| 79 ++-
4 files changed, 179 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/455e3c1b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveQL.java
--
diff --git
a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveQL.java
b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveQL.java
index e053a9a..bb5d526 100644
---
a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveQL.java
+++
b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveQL.java
@@ -149,10 +149,11 @@ public class PutHiveQL extends AbstractHiveQLProcessor {
} else if (e instanceof SQLException) {
// Use the SQLException's vendor code for guidance -- see
Hive's ErrorMsg class for details on error codes
int errorCode = ((SQLException) e).getErrorCode();
+getLogger().debug("Error occurred during Hive operation, Hive
returned error code {}", new Object[]{errorCode});
if (errorCode >= 1 && errorCode < 2) {
return ErrorTypes.InvalidInput;
} else if (errorCode >= 2 && errorCode < 3) {
-return ErrorTypes.TemporalFailure;
+return ErrorTypes.InvalidInput;
} else if (errorCode >= 3 && errorCode < 4) {
return ErrorTypes.TemporalInputFailure;
} else if (errorCode >= 4 && errorCode < 5) {
@@ -160,7 +161,9 @@ public class PutHiveQL extends AbstractHiveQLProcessor {
// a ProcessException, we'll route to failure via an
InvalidInput error type.
return ErrorTypes.InvalidInput;
} else {
-return ErrorTypes.UnknownFailure;
+// Default unknown errors to TemporalFailure (as they were
implemented originally), so they can be routed to failure
+// or rolled back depending on the user's setting of
Rollback On Failure.
+return ErrorTypes.TemporalFailure;
}
} else {
return ErrorTypes.UnknownFailure;
http://git-wip-us.apache.org/repos/asf/nifi/blob/455e3c1b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/test/java/org/apache/nifi/processors/hive/TestPutHiveQL.java
--
diff --git
a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/test/java/org/apache/nifi/processors/hive/TestPutHiveQL.java
b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/test/java/org/apache/nifi/processors/hive/TestPutHiveQL.java
index af737ae..dd16ca1 100644
---
a/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/test/java/org/apache/nifi/processors/hive/TestPutHiveQL.java
+++
b/nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/test/java/org/apache/nifi/processors/hive/TestPutHiveQL.java
@@ -719,6 +719,78 @@ public class TestPutHiveQL {
runner.assertAllFlowFilesTransferred(PutHiveQL.REL_RETRY, 0);
}
+@Test
+public void testUnknownFailure() throws InitializationException,
ProcessException {
+final TestRunner runner = TestRunners.newTestRunner(PutHiveQL.class);
+final SQLExceptionService service = new SQLExceptionService(null);
+service.setErrorCode(2);
+runner.addControllerService("dbcp", service);
+runner.enableControllerService(service);
+
+runner.setProperty(PutHiveQL.HIVE_DBCP_SERVICE, "dbcp");
+
+fin
