[GitHub] [nifi] r-sidd commented on pull request #7341: NIFI-11643: Upgrade geoip2 to 4.0.1
r-sidd commented on PR #7341: URL: https://github.com/apache/nifi/pull/7341#issuecomment-1577883822 > Version 4.0.1 includes changes from [3.0.0](https://github.com/maxmind/GeoIP2-java/releases/tag/v3.0.0) that require Java 11 as the minimum version, so this should be suitable for inclusion on the main branch, but cannot be backported to the version 1 support branch. Cool, how should we go about it 🙂 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
dan-s1 commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218681571
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
I simplified the RowIterator by doing some up front processing in the
constructor to determine which sheets need to be processed. That allowed for
simplifying later logic to be uniform just to act on the the sheets needed. I
hope its okay that I have a call to `setCurrent()` in the constructor in order
to initialize the `currentRow` to allow `hasNext` to be called initially.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
dan-s1 commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218681571
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
I simplified the RowIterator by doing some up front processing in the
constructor to determine which sheets need to be processed. That allowed for
simplifying later logic to be uniform just to act on the the sheets needed. I
hope its okay that I have a call to `setCurrent()` in the constructor in order
to initialize the `currentRow` to allow hasNext to be called initially.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mattyb149 closed pull request #7343: NIFI-11646: Deprecate Lua and Ruby script engines
mattyb149 closed pull request #7343: NIFI-11646: Deprecate Lua and Ruby script engines URL: https://github.com/apache/nifi/pull/7343 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11641) Upgrade Netty to 4.1.93
[ https://issues.apache.org/jira/browse/NIFI-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11641: Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Netty to 4.1.93 > --- > > Key: NIFI-11641 > URL: https://issues.apache.org/jira/browse/NIFI-11641 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Netty [4.1.93|https://netty.io/news/2023/05/25/4-1-93-Final.html] includes > several minor bug fixes and improvements, several related to HTTP/2 handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11645) Upgrade Guava to 32.0.0
[ https://issues.apache.org/jira/browse/NIFI-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11645: Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Guava to 32.0.0 > --- > > Key: NIFI-11645 > URL: https://issues.apache.org/jira/browse/NIFI-11645 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Multiple framework and extension components have dependencies on Google > Guava. Most of these dependencies depend on version 31.1. > Google Guava [32.0.0|https://github.com/google/guava/releases/tag/v32.0.0] > maintains binary compatibility with earlier versions and also resolves > CVE-2020-8908 and CVE-2023-2976 related to permissions on created temporary > directories. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11646) Deprecate Lua and Ruby Script Engines
[
https://issues.apache.org/jira/browse/NIFI-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-11646:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Deprecate Lua and Ruby Script Engines
> -
>
> Key: NIFI-11646
> URL: https://issues.apache.org/jira/browse/NIFI-11646
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.22.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{lua}} and {{ruby}} Script Engines for multiple scripted Processors and
> Controller Services should be deprecated for removal in NiFi 2.0. The engines
> are not often used and as they are not JVM-native languages the capabilities
> for these engines are more limited than a JVM-native language such as Groovy.
> This Jira continues the idea of NIFI-11630 but for the Lua and Ruby script
> engines.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11645) Upgrade Guava to 32.0.0
[ https://issues.apache.org/jira/browse/NIFI-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729511#comment-17729511 ] ASF subversion and git services commented on NIFI-11645: Commit 691da641ee27dae2d04da3f7b77bf62ae0118dd2 in nifi's branch refs/heads/support/nifi-1.x from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=691da641ee ] NIFI-11645 correct mistaken accumulo version ref after pulling guava changes in > Upgrade Guava to 32.0.0 > --- > > Key: NIFI-11645 > URL: https://issues.apache.org/jira/browse/NIFI-11645 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Multiple framework and extension components have dependencies on Google > Guava. Most of these dependencies depend on version 31.1. > Google Guava [32.0.0|https://github.com/google/guava/releases/tag/v32.0.0] > maintains binary compatibility with earlier versions and also resolves > CVE-2020-8908 and CVE-2023-2976 related to permissions on created temporary > directories. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11641) Upgrade Netty to 4.1.93
[ https://issues.apache.org/jira/browse/NIFI-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729506#comment-17729506 ] ASF subversion and git services commented on NIFI-11641: Commit 8278f0a3bf1816a47bafe586e81d13f7a39d9efc in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=8278f0a3bf ] NIFI-11641 This closes #7340. Upgraded Netty from 4.1.92 to 4.1.93 Signed-off-by: Joe Witt > Upgrade Netty to 4.1.93 > --- > > Key: NIFI-11641 > URL: https://issues.apache.org/jira/browse/NIFI-11641 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Netty [4.1.93|https://netty.io/news/2023/05/25/4-1-93-Final.html] includes > several minor bug fixes and improvements, several related to HTTP/2 handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11646) Deprecate Lua and Ruby Script Engines
[
https://issues.apache.org/jira/browse/NIFI-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729508#comment-17729508
]
ASF subversion and git services commented on NIFI-11646:
Commit ea35e61333b119415a5fdc91d232ba63ce1463e7 in nifi's branch
refs/heads/support/nifi-1.x from Matt Burgess
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ea35e61333 ]
NIFI-11646: This closes #7343. Deprecate Lua and Ruby script engines
Signed-off-by: Joe Witt
> Deprecate Lua and Ruby Script Engines
> -
>
> Key: NIFI-11646
> URL: https://issues.apache.org/jira/browse/NIFI-11646
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.22.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{lua}} and {{ruby}} Script Engines for multiple scripted Processors and
> Controller Services should be deprecated for removal in NiFi 2.0. The engines
> are not often used and as they are not JVM-native languages the capabilities
> for these engines are more limited than a JVM-native language such as Groovy.
> This Jira continues the idea of NIFI-11630 but for the Lua and Ruby script
> engines.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11645) Upgrade Guava to 32.0.0
[ https://issues.apache.org/jira/browse/NIFI-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729507#comment-17729507 ] ASF subversion and git services commented on NIFI-11645: Commit 6d00ce4fe5f0082d16fba26148c989b81341ab6a in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6d00ce4fe5 ] NIFI-11645 This closes #7342. Upgraded Guava from 31.1 to 32.0.0 Signed-off-by: Joe Witt > Upgrade Guava to 32.0.0 > --- > > Key: NIFI-11645 > URL: https://issues.apache.org/jira/browse/NIFI-11645 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Multiple framework and extension components have dependencies on Google > Guava. Most of these dependencies depend on version 31.1. > Google Guava [32.0.0|https://github.com/google/guava/releases/tag/v32.0.0] > maintains binary compatibility with earlier versions and also resolves > CVE-2020-8908 and CVE-2023-2976 related to permissions on created temporary > directories. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11160) Upgrade Apache Tika to 2.7.0
[ https://issues.apache.org/jira/browse/NIFI-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729509#comment-17729509 ] ASF subversion and git services commented on NIFI-11160: Commit 52e2f0ccd6217fa91f7c5214ccc7fd941d4f86fc in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=52e2f0ccd6 ] NIFI-11160 Upgraded Apache Tika from 2.6.0 to 2.7.0 Signed-off-by: Pierre Villard This closes #6938. > Upgrade Apache Tika to 2.7.0 > > > Key: NIFI-11160 > URL: https://issues.apache.org/jira/browse/NIFI-11160 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Apache Tika > [2.7.0|https://dist.apache.org/repos/dist/release/tika/2.7.0/CHANGES-2.7.0.txt] > includes several bug fixes and transitive dependency upgrades. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11548) Upgrade Apache Tika to 2.8.0
[ https://issues.apache.org/jira/browse/NIFI-11548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729510#comment-17729510 ] ASF subversion and git services commented on NIFI-11548: Commit 580b3a61aa44b4125af7a96406dce90b44f6acf6 in nifi's branch refs/heads/support/nifi-1.x from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=580b3a61aa ] NIFI-11548 Upgraded Apache Tika from 2.7.0 to 2.8.0 Signed-off-by: Pierre Villard This closes #7247. > Upgrade Apache Tika to 2.8.0 > > > Key: NIFI-11548 > URL: https://issues.apache.org/jira/browse/NIFI-11548 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > > [Apache Tika 2.8.0|https://tika.apache.org/2.8.0/index.html] includes a > number of bug fixes and type detection improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1572: MINIFICPP-2027 Upgrade Google Cloud library to version 2.10.1
szaszm commented on code in PR #1572:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1572#discussion_r1218659254
##
cmake/GoogleCloudCpp.cmake:
##
@@ -22,17 +22,36 @@ include(Crc32c)
set(PATCH_FILE_1
"${CMAKE_SOURCE_DIR}/thirdparty/google-cloud-cpp/remove-find_package.patch")
set(PATCH_FILE_2
"${CMAKE_SOURCE_DIR}/thirdparty/google-cloud-cpp/nlohmann_lib_as_interface.patch")
-set(PC ${Bash_EXECUTABLE} -c "set -x &&\
-(\\\"${Patch_EXECUTABLE}\\\" -p1 -R -s -f --dry-run -i
\\\"${PATCH_FILE_1}\\\" || \\\"${Patch_EXECUTABLE}\\\" -p1 -N -i
\\\"${PATCH_FILE_1}\\\") &&\
-(\\\"${Patch_EXECUTABLE}\\\" -p1 -R -s -f --dry-run -i
\\\"${PATCH_FILE_2}\\\" || \\\"${Patch_EXECUTABLE}\\\" -p1 -N -i
\\\"${PATCH_FILE_2}\\\")")
+set(PATCH_FILE_3
"${CMAKE_SOURCE_DIR}/thirdparty/google-cloud-cpp/mock-client-without-decorators.patch")
+set(PATCH_FILE_4
"${CMAKE_SOURCE_DIR}/thirdparty/google-cloud-cpp/mock_client_target.patch")
+if (SKIP_TESTS)
+set(PC ${Bash_EXECUTABLE} -c "set -x &&\
+(\\\"${Patch_EXECUTABLE}\\\" -p1 -R -s -f --dry-run -i
\\\"${PATCH_FILE_1}\\\" || \\\"${Patch_EXECUTABLE}\\\" -p1 -N -i
\\\"${PATCH_FILE_1}\\\") &&\
+(\\\"${Patch_EXECUTABLE}\\\" -p1 -R -s -f --dry-run -i
\\\"${PATCH_FILE_2}\\\" || \\\"${Patch_EXECUTABLE}\\\" -p1 -N -i
\\\"${PATCH_FILE_2}\\\")")
Review Comment:
I meant single escaping, not single quotes. Do they work? It's like that in
most other places, although there are a few more double escaped instances as
well.
```suggestion
(\"${Patch_EXECUTABLE}\" -p1 -R -s -f --dry-run -i
\"${PATCH_FILE_1}\" || \"${Patch_EXECUTABLE}\" -p1 -N -i \"${PATCH_FILE_1}\")
&&\
(\"${Patch_EXECUTABLE}\" -p1 -R -s -f --dry-run -i
\"${PATCH_FILE_2}\" || \"${Patch_EXECUTABLE}\" -p1 -N -i \"${PATCH_FILE_2}\")")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
dan-s1 commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218655660
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
I am reworking it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-11420) Upgrade ActiveMQ to 5.18
[ https://issues.apache.org/jira/browse/NIFI-11420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11420: Fix Version/s: (was: 1.22.0) > Upgrade ActiveMQ to 5.18 > > > Key: NIFI-11420 > URL: https://issues.apache.org/jira/browse/NIFI-11420 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade ActiveMQ 5.15.15 to 5.18.0 > [https://activemq.apache.org/activemq-5018000-release] > [https://activemq.apache.org/activemq-5017000-release] > [https://activemq.apache.org/activemq-5016000-release] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11646) Deprecate Lua and Ruby Script Engines
[
https://issues.apache.org/jira/browse/NIFI-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-11646:
Fix Version/s: 1.22.0
(was: 1.latest)
> Deprecate Lua and Ruby Script Engines
> -
>
> Key: NIFI-11646
> URL: https://issues.apache.org/jira/browse/NIFI-11646
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.22.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{lua}} and {{ruby}} Script Engines for multiple scripted Processors and
> Controller Services should be deprecated for removal in NiFi 2.0. The engines
> are not often used and as they are not JVM-native languages the capabilities
> for these engines are more limited than a JVM-native language such as Groovy.
> This Jira continues the idea of NIFI-11630 but for the Lua and Ruby script
> engines.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11641) Upgrade Netty to 4.1.93
[ https://issues.apache.org/jira/browse/NIFI-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11641: Fix Version/s: 2.0.0 1.22.0 (was: 1.latest) (was: 2.latest) > Upgrade Netty to 4.1.93 > --- > > Key: NIFI-11641 > URL: https://issues.apache.org/jira/browse/NIFI-11641 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Netty [4.1.93|https://netty.io/news/2023/05/25/4-1-93-Final.html] includes > several minor bug fixes and improvements, several related to HTTP/2 handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11645) Upgrade Guava to 32.0.0
[ https://issues.apache.org/jira/browse/NIFI-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11645: Fix Version/s: 2.0.0 1.22.0 (was: 1.latest) (was: 2.latest) > Upgrade Guava to 32.0.0 > --- > > Key: NIFI-11645 > URL: https://issues.apache.org/jira/browse/NIFI-11645 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Multiple framework and extension components have dependencies on Google > Guava. Most of these dependencies depend on version 31.1. > Google Guava [32.0.0|https://github.com/google/guava/releases/tag/v32.0.0] > maintains binary compatibility with earlier versions and also resolves > CVE-2020-8908 and CVE-2023-2976 related to permissions on created temporary > directories. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11420) Upgrade ActiveMQ to 5.18
[ https://issues.apache.org/jira/browse/NIFI-11420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11420: Fix Version/s: 1.22.0 > Upgrade ActiveMQ to 5.18 > > > Key: NIFI-11420 > URL: https://issues.apache.org/jira/browse/NIFI-11420 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 2.0.0, 1.22.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrade ActiveMQ 5.15.15 to 5.18.0 > [https://activemq.apache.org/activemq-5018000-release] > [https://activemq.apache.org/activemq-5017000-release] > [https://activemq.apache.org/activemq-5016000-release] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11548) Upgrade Apache Tika to 2.8.0
[ https://issues.apache.org/jira/browse/NIFI-11548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11548: Fix Version/s: 1.22.0 > Upgrade Apache Tika to 2.8.0 > > > Key: NIFI-11548 > URL: https://issues.apache.org/jira/browse/NIFI-11548 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > > [Apache Tika 2.8.0|https://tika.apache.org/2.8.0/index.html] includes a > number of bug fixes and type detection improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11160) Upgrade Apache Tika to 2.7.0
[ https://issues.apache.org/jira/browse/NIFI-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11160: Fix Version/s: 1.22.0 > Upgrade Apache Tika to 2.7.0 > > > Key: NIFI-11160 > URL: https://issues.apache.org/jira/browse/NIFI-11160 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Apache Tika > [2.7.0|https://dist.apache.org/repos/dist/release/tika/2.7.0/CHANGES-2.7.0.txt] > includes several bug fixes and transitive dependency upgrades. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11647) org.apache.nifi.serialization.record.util.DataTypeUtils.getSQLTypeValue does not map UUID RecordFieldType
[
https://issues.apache.org/jira/browse/NIFI-11647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess reassigned NIFI-11647:
---
Assignee: Matt Burgess
> org.apache.nifi.serialization.record.util.DataTypeUtils.getSQLTypeValue does
> not map UUID RecordFieldType
> -
>
> Key: NIFI-11647
> URL: https://issues.apache.org/jira/browse/NIFI-11647
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Sander Bylemans
>Assignee: Matt Burgess
>Priority: Major
>
> org.apache.nifi.serialization.record.util.DataTypeUtils.getSQLTypeValue does
> not map the RecordFieldType UUID.
> This causes the PutDatabaseRecord to fail when trying to INSERT a flowfile
> with a UUID as logical type with the error message:
> {code}
> 2023-06-05 22:03:29,872 ERROR [Timer-Driven Process Thread-11]
> o.a.n.p.standard.PutDatabaseRecord
> PutDatabaseRecord[id=8f505e85-8058-3714-ac24-aaeeb5efc6a3] Failed to put
> Records to database for
> StandardFlowFileRecord[uuid=cedad728-117a-4235-9251-ded3b7580b7b,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1685995389355-150,
> container=default, section=150], offset=2643,
> length=6551],offset=0,name=fase_3.2.23_00699164_00699164.parquet,size=4991].
> Routing to failure.
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert unknown type UUID
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.getSQLTypeValue(DataTypeUtils.java:2148)
> at
> org.apache.nifi.processors.standard.PutDatabaseRecord.executeDML(PutDatabaseRecord.java:723)
> at
> org.apache.nifi.processors.standard.PutDatabaseRecord.putToDatabase(PutDatabaseRecord.java:970)
> at
> org.apache.nifi.processors.standard.PutDatabaseRecord.onTrigger(PutDatabaseRecord.java:493)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1356)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
> Possibly more are affected.
> This was added in https://issues.apache.org/jira/browse/NIFI-9981 and there
> was already concern for the need of mapping a UUID.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11645) Upgrade Guava to 32.0.0
[ https://issues.apache.org/jira/browse/NIFI-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729485#comment-17729485 ] ASF subversion and git services commented on NIFI-11645: Commit c5c711c91d4dcca34d7b989037f4e748b402b8d2 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c5c711c91d ] NIFI-11645 This closes #7342. Upgraded Guava from 31.1 to 32.0.0 - Added selective overrides to upgrade transitive dependencies Signed-off-by: Joe Witt > Upgrade Guava to 32.0.0 > --- > > Key: NIFI-11645 > URL: https://issues.apache.org/jira/browse/NIFI-11645 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > Multiple framework and extension components have dependencies on Google > Guava. Most of these dependencies depend on version 31.1. > Google Guava [32.0.0|https://github.com/google/guava/releases/tag/v32.0.0] > maintains binary compatibility with earlier versions and also resolves > CVE-2020-8908 and CVE-2023-2976 related to permissions on created temporary > directories. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11641) Upgrade Netty to 4.1.93
[ https://issues.apache.org/jira/browse/NIFI-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729484#comment-17729484 ] ASF subversion and git services commented on NIFI-11641: Commit 2a9f87a1e5cc1e3035446696d6bfc2a7c6ee799a in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2a9f87a1e5 ] NIFI-11641 This closes #7340. Upgraded Netty from 4.1.92 to 4.1.93 Signed-off-by: Joe Witt > Upgrade Netty to 4.1.93 > --- > > Key: NIFI-11641 > URL: https://issues.apache.org/jira/browse/NIFI-11641 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > Netty [4.1.93|https://netty.io/news/2023/05/25/4-1-93-Final.html] includes > several minor bug fixes and improvements, several related to HTTP/2 handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11549) Add Azure Queue Storage Processors using Azure SDK 12
[
https://issues.apache.org/jira/browse/NIFI-11549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-11549.
-
Fix Version/s: 2.0.0
1.22.0
(was: 2.latest)
Resolution: Fixed
> Add Azure Queue Storage Processors using Azure SDK 12
> -
>
> Key: NIFI-11549
> URL: https://issues.apache.org/jira/browse/NIFI-11549
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: David Handermann
>Assignee: Emilio Setiadarma
>Priority: Major
> Fix For: 2.0.0, 1.22.0
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> The {{GetAzureQueueStorage}} and {{PutAzureQueueStorage}} Processors depend
> on the legacy Microsoft Azure SDK libraries. The current implementations also
> depend on the Storage Credentials Service based on the legacy SDK.
> Following the pattern of the Azure Blob Storage Processors, new
> implementations should be created that depend on the current Azure SDK
> version 12. Adding new Processors will enable subsequent deprecation of the
> current versions. The new Processors should leverage the Credentials Service
> with Azure SDK 12.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11549) Add Azure Queue Storage Processors using Azure SDK 12
[
https://issues.apache.org/jira/browse/NIFI-11549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729483#comment-17729483
]
ASF subversion and git services commented on NIFI-11549:
Commit a963434a8c8c20e7a11fa4b9952880a08ca52cc6 in nifi's branch
refs/heads/support/nifi-1.x from Emilio Setiadarma
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a963434a8c ]
NIFI-11549 Added AzureQueueStorage_v12 Processors
- Deprecated GetAzureQueueStorage and PutAzureQueueStorage
This closes #7269
Signed-off-by: David Handermann
(cherry picked from commit b11373af7b017f83c3b579c6aefc7cb7a1816bfd)
> Add Azure Queue Storage Processors using Azure SDK 12
> -
>
> Key: NIFI-11549
> URL: https://issues.apache.org/jira/browse/NIFI-11549
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: David Handermann
>Assignee: Emilio Setiadarma
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> The {{GetAzureQueueStorage}} and {{PutAzureQueueStorage}} Processors depend
> on the legacy Microsoft Azure SDK libraries. The current implementations also
> depend on the Storage Credentials Service based on the legacy SDK.
> Following the pattern of the Azure Blob Storage Processors, new
> implementations should be created that depend on the current Azure SDK
> version 12. Adding new Processors will enable subsequent deprecation of the
> current versions. The new Processors should leverage the Credentials Service
> with Azure SDK 12.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11567) GeoEnrichIP processors should auto-reload the database file
[ https://issues.apache.org/jira/browse/NIFI-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11567: Fix Version/s: 2.0.0 1.22.0 (was: 1.latest) > GeoEnrichIP processors should auto-reload the database file > --- > > Key: NIFI-11567 > URL: https://issues.apache.org/jira/browse/NIFI-11567 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 2.0.0, 1.22.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the GeoEnrichIP processors only load the database when the > processor is scheduled. This requires a processor restart if the database > file changes. Instead, the processors should auto-reload the database file > when it detects a change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11567) GeoEnrichIP processors should auto-reload the database file
[ https://issues.apache.org/jira/browse/NIFI-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11567: Resolution: Fixed Status: Resolved (was: Patch Available) > GeoEnrichIP processors should auto-reload the database file > --- > > Key: NIFI-11567 > URL: https://issues.apache.org/jira/browse/NIFI-11567 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 2.0.0, 1.22.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the GeoEnrichIP processors only load the database when the > processor is scheduled. This requires a processor restart if the database > file changes. Instead, the processors should auto-reload the database file > when it detects a change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11567) GeoEnrichIP processors should auto-reload the database file
[ https://issues.apache.org/jira/browse/NIFI-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11567: Fix Version/s: (was: 2.latest) > GeoEnrichIP processors should auto-reload the database file > --- > > Key: NIFI-11567 > URL: https://issues.apache.org/jira/browse/NIFI-11567 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the GeoEnrichIP processors only load the database when the > processor is scheduled. This requires a processor restart if the database > file changes. Instead, the processors should auto-reload the database file > when it detects a change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11647) org.apache.nifi.serialization.record.util.DataTypeUtils.getSQLTypeValue does not map UUID RecordFieldType
Sander Bylemans created NIFI-11647:
--
Summary:
org.apache.nifi.serialization.record.util.DataTypeUtils.getSQLTypeValue does
not map UUID RecordFieldType
Key: NIFI-11647
URL: https://issues.apache.org/jira/browse/NIFI-11647
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Sander Bylemans
org.apache.nifi.serialization.record.util.DataTypeUtils.getSQLTypeValue does
not map the RecordFieldType UUID.
This causes the PutDatabaseRecord to fail when trying to INSERT a flowfile with
a UUID as logical type with the error message:
{code}
2023-06-05 22:03:29,872 ERROR [Timer-Driven Process Thread-11]
o.a.n.p.standard.PutDatabaseRecord
PutDatabaseRecord[id=8f505e85-8058-3714-ac24-aaeeb5efc6a3] Failed to put
Records to database for
StandardFlowFileRecord[uuid=cedad728-117a-4235-9251-ded3b7580b7b,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1685995389355-150, container=default,
section=150], offset=2643,
length=6551],offset=0,name=fase_3.2.23_00699164_00699164.parquet,size=4991].
Routing to failure.
org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
Cannot convert unknown type UUID
at
org.apache.nifi.serialization.record.util.DataTypeUtils.getSQLTypeValue(DataTypeUtils.java:2148)
at
org.apache.nifi.processors.standard.PutDatabaseRecord.executeDML(PutDatabaseRecord.java:723)
at
org.apache.nifi.processors.standard.PutDatabaseRecord.putToDatabase(PutDatabaseRecord.java:970)
at
org.apache.nifi.processors.standard.PutDatabaseRecord.onTrigger(PutDatabaseRecord.java:493)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1356)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
Possibly more are affected.
This was added in https://issues.apache.org/jira/browse/NIFI-9981 and there was
already concern for the need of mapping a UUID.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11549) Add Azure Queue Storage Processors using Azure SDK 12
[
https://issues.apache.org/jira/browse/NIFI-11549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729481#comment-17729481
]
ASF subversion and git services commented on NIFI-11549:
Commit b11373af7b017f83c3b579c6aefc7cb7a1816bfd in nifi's branch
refs/heads/main from Emilio Setiadarma
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b11373af7b ]
NIFI-11549 Added AzureQueueStorage_v12 Processors
- Deprecated GetAzureQueueStorage and PutAzureQueueStorage
This closes #7269
Signed-off-by: David Handermann
> Add Azure Queue Storage Processors using Azure SDK 12
> -
>
> Key: NIFI-11549
> URL: https://issues.apache.org/jira/browse/NIFI-11549
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: David Handermann
>Assignee: Emilio Setiadarma
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> The {{GetAzureQueueStorage}} and {{PutAzureQueueStorage}} Processors depend
> on the legacy Microsoft Azure SDK libraries. The current implementations also
> depend on the Storage Credentials Service based on the legacy SDK.
> Following the pattern of the Azure Blob Storage Processors, new
> implementations should be created that depend on the current Azure SDK
> version 12. Adding new Processors will enable subsequent deprecation of the
> current versions. The new Processors should leverage the Credentials Service
> with Azure SDK 12.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #7269: NIFI-11549: implemented AzureQueueStorage_v12 processors
exceptionfactory closed pull request #7269: NIFI-11549: implemented AzureQueueStorage_v12 processors URL: https://github.com/apache/nifi/pull/7269 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11646) Deprecate Lua and Ruby Script Engines
[
https://issues.apache.org/jira/browse/NIFI-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-11646:
Status: Patch Available (was: In Progress)
> Deprecate Lua and Ruby Script Engines
> -
>
> Key: NIFI-11646
> URL: https://issues.apache.org/jira/browse/NIFI-11646
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.latest
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{lua}} and {{ruby}} Script Engines for multiple scripted Processors and
> Controller Services should be deprecated for removal in NiFi 2.0. The engines
> are not often used and as they are not JVM-native languages the capabilities
> for these engines are more limited than a JVM-native language such as Groovy.
> This Jira continues the idea of NIFI-11630 but for the Lua and Ruby script
> engines.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] mattyb149 opened a new pull request, #7343: NIFI-11646: Deprecate Lua and Ruby script engines
mattyb149 opened a new pull request, #7343: URL: https://github.com/apache/nifi/pull/7343 # Summary [NIFI-11646](https://issues.apache.org/jira/browse/NIFI-11646) This PR follows [NIFI-11630](https://issues.apache.org/jira/browse/NIFI-11630) but deprecates the Lua and Ruby script engines which are slated for removal from NiFi 2.0. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [x] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [x] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-11593) UI - Dependent Properties with No Allowable Values
[ https://issues.apache.org/jira/browse/NIFI-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Margot Tien resolved NIFI-11593. Resolution: Fixed > UI - Dependent Properties with No Allowable Values > -- > > Key: NIFI-11593 > URL: https://issues.apache.org/jira/browse/NIFI-11593 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Shane Ardell >Priority: Major > Fix For: 2.0.0, 1.22.0 > > Time Spent: 50m > Remaining Estimate: 0h > > The front end does not support dependent properties correctly when the > underlying Property does not have allowable values. When the dependent > property does not have allowable values, any properties that depend on it > should be shown when it is set (contains any value). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11646) Deprecate Lua and Ruby Script Engines
[
https://issues.apache.org/jira/browse/NIFI-11646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess reassigned NIFI-11646:
---
Assignee: Matt Burgess
> Deprecate Lua and Ruby Script Engines
> -
>
> Key: NIFI-11646
> URL: https://issues.apache.org/jira/browse/NIFI-11646
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.latest
>
>
> The {{lua}} and {{ruby}} Script Engines for multiple scripted Processors and
> Controller Services should be deprecated for removal in NiFi 2.0. The engines
> are not often used and as they are not JVM-native languages the capabilities
> for these engines are more limited than a JVM-native language such as Groovy.
> This Jira continues the idea of NIFI-11630 but for the Lua and Ruby script
> engines.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-11646) Deprecate Lua and Ruby Script Engines
Matt Burgess created NIFI-11646:
---
Summary: Deprecate Lua and Ruby Script Engines
Key: NIFI-11646
URL: https://issues.apache.org/jira/browse/NIFI-11646
Project: Apache NiFi
Issue Type: Task
Components: Extensions
Reporter: Matt Burgess
Fix For: 1.latest
The {{lua}} and {{ruby}} Script Engines for multiple scripted Processors and
Controller Services should be deprecated for removal in NiFi 2.0. The engines
are not often used and as they are not JVM-native languages the capabilities
for these engines are more limited than a JVM-native language such as Groovy.
This Jira continues the idea of NIFI-11630 but for the Lua and Ruby script
engines.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11593) UI - Dependent Properties with No Allowable Values
[ https://issues.apache.org/jira/browse/NIFI-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729469#comment-17729469 ] ASF subversion and git services commented on NIFI-11593: Commit 70bb41ec95ef44561bc8a1ed570a080ce7a5354d in nifi's branch refs/heads/support/nifi-1.x from Shane Ardell [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=70bb41ec95 ] NIFI-11593: Fix hiding logic for properties (#7339) * NIFI-11593: fix hiding logic for properties with references or without dependent values * NIFI-11593: remove arrow functions Merged #7339 into main. > UI - Dependent Properties with No Allowable Values > -- > > Key: NIFI-11593 > URL: https://issues.apache.org/jira/browse/NIFI-11593 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Shane Ardell >Priority: Major > Fix For: 2.0.0, 1.22.0 > > Time Spent: 50m > Remaining Estimate: 0h > > The front end does not support dependent properties correctly when the > underlying Property does not have allowable values. When the dependent > property does not have allowable values, any properties that depend on it > should be shown when it is set (contains any value). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
exceptionfactory commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218567911
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
Although it may not be the most intuitive approach, having `hasNext()` throw
an `UnsupportedOperationException` and changing callers to just use `next()`
could work, although it raises a question of logic handling for the calling
class. Another option could be folding the functionality into the calling class.
I think it would be cleaner in the end to rework the RowIterator, in
addition to removing the `public` visibility from the class. I could take a
look at reworking RowIterator and pushing the changes to this branch if you
think the changes are too involved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11593) UI - Dependent Properties with No Allowable Values
[ https://issues.apache.org/jira/browse/NIFI-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729467#comment-17729467 ] ASF subversion and git services commented on NIFI-11593: Commit 7a7f6f622961ab494af41f26259d08f0906c2183 in nifi's branch refs/heads/main from Shane Ardell [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7a7f6f6229 ] NIFI-11593: Fix hiding logic for properties (#7339) * NIFI-11593: fix hiding logic for properties with references or without dependent values * NIFI-11593: remove arrow functions Merged #7339 into main. > UI - Dependent Properties with No Allowable Values > -- > > Key: NIFI-11593 > URL: https://issues.apache.org/jira/browse/NIFI-11593 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Shane Ardell >Priority: Major > Fix For: 2.0.0, 1.22.0 > > Time Spent: 50m > Remaining Estimate: 0h > > The front end does not support dependent properties correctly when the > underlying Property does not have allowable values. When the dependent > property does not have allowable values, any properties that depend on it > should be shown when it is set (contains any value). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mtien-apache merged pull request #7339: NIFI-11593: Fix hiding logic for properties
mtien-apache merged PR #7339: URL: https://github.com/apache/nifi/pull/7339 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
dan-s1 commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218557537
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
It looks like I could take out lines
```
currentSheet = null;
currentRows = null;
```
from setCurrent without any issues. Since the RowIterator is for internal
use, could we just implement the next method and not the hasNext method? It
seem there would be considerable amount of change needed just to have the
hasNext method work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-11643) Upgrade geoip2 to 4.0.1
[ https://issues.apache.org/jira/browse/NIFI-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11643: Fix Version/s: 2.latest > Upgrade geoip2 to 4.0.1 > --- > > Key: NIFI-11643 > URL: https://issues.apache.org/jira/browse/NIFI-11643 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > > Bump version from 2.16.1 to 4.0.1 to remediate findings: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-42004] > [https://mvnrepository.com/artifact/com.maxmind.geoip2/geoip2/2.16.1#:~:text=CVE%2D2022%2D42004-,CVE%2D2022%2D42003,-CVE%2D2021%2D46877] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-46877] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11643) Upgrade geoip2 to 4.0.1
[ https://issues.apache.org/jira/browse/NIFI-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11643: Affects Version/s: (was: 2.0.0) (was: 1.21.0) > Upgrade geoip2 to 4.0.1 > --- > > Key: NIFI-11643 > URL: https://issues.apache.org/jira/browse/NIFI-11643 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Time Spent: 20m > Remaining Estimate: 0h > > Bump version from 2.16.1 to 4.0.1 to remediate findings: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-42004] > [https://mvnrepository.com/artifact/com.maxmind.geoip2/geoip2/2.16.1#:~:text=CVE%2D2022%2D42004-,CVE%2D2022%2D42003,-CVE%2D2021%2D46877] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-46877] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11643) Upgrade geoip2 to 4.0.1
[ https://issues.apache.org/jira/browse/NIFI-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729461#comment-17729461 ] David Handermann commented on NIFI-11643: - [Version 3.0.0|https://github.com/maxmind/GeoIP2-java/releases/tag/v3.0.0] requires Java 11, so this upgrade is suitable for inclusion in NiFi 2.0.0 on the main branch, but cannot be backported to the NiFi 1 support branch. > Upgrade geoip2 to 4.0.1 > --- > > Key: NIFI-11643 > URL: https://issues.apache.org/jira/browse/NIFI-11643 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.latest > > Time Spent: 20m > Remaining Estimate: 0h > > Bump version from 2.16.1 to 4.0.1 to remediate findings: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-42004] > [https://mvnrepository.com/artifact/com.maxmind.geoip2/geoip2/2.16.1#:~:text=CVE%2D2022%2D42004-,CVE%2D2022%2D42003,-CVE%2D2021%2D46877] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-46877] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11645) Upgrade Guava to 32.0.0
[ https://issues.apache.org/jira/browse/NIFI-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11645: Status: Patch Available (was: Open) > Upgrade Guava to 32.0.0 > --- > > Key: NIFI-11645 > URL: https://issues.apache.org/jira/browse/NIFI-11645 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > Multiple framework and extension components have dependencies on Google > Guava. Most of these dependencies depend on version 31.1. > Google Guava [32.0.0|https://github.com/google/guava/releases/tag/v32.0.0] > maintains binary compatibility with earlier versions and also resolves > CVE-2020-8908 and CVE-2023-2976 related to permissions on created temporary > directories. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7342: NIFI-11645 Upgrade Guava from 31.1 to 32.0.0
exceptionfactory opened a new pull request, #7342: URL: https://github.com/apache/nifi/pull/7342 # Summary [NIFI-11645](https://issues.apache.org/jira/browse/NIFI-11645) Upgrades multiple direct and transitive references to Google Guava from 31.1 to 32.0.0. [Guava 32.0.0](https://github.com/google/guava/releases/tag/v32.0.0) maintains binary compatibility with earlier versions and addresses CVE-2020-8908 and [CVE-2023-2976](https://ossindex.sonatype.org/vulnerability/CVE-2023-2976). # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11645) Upgrade Guava to 32.0.0
David Handermann created NIFI-11645: --- Summary: Upgrade Guava to 32.0.0 Key: NIFI-11645 URL: https://issues.apache.org/jira/browse/NIFI-11645 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: David Handermann Assignee: David Handermann Fix For: 1.latest, 2.latest Multiple framework and extension components have dependencies on Google Guava. Most of these dependencies depend on version 31.1. Google Guava [32.0.0|https://github.com/google/guava/releases/tag/v32.0.0] maintains binary compatibility with earlier versions and also resolves CVE-2020-8908 and CVE-2023-2976 related to permissions on created temporary directories. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11644) Upgrade Hazelcast to 5.3.0
[ https://issues.apache.org/jira/browse/NIFI-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-11644. - Fix Version/s: 2.0.0 1.22.0 Resolution: Fixed > Upgrade Hazelcast to 5.3.0 > -- > > Key: NIFI-11644 > URL: https://issues.apache.org/jira/browse/NIFI-11644 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 20m > Remaining Estimate: 0h > > [Hazelcast 5.3.0|https://github.com/hazelcast/hazelcast/releases/tag/v5.3.0] > includes a number of new features and improvements while maintaining > compatibility with earlier releases. > The [Hazelcast Compatibility > Guarantees|https://docs.hazelcast.com/hazelcast/5.3/deploy/versioning-compatibility#compatibility-guarantees] > documentation notes that the Java client version 5 is compatible with both > Hazelcast 4 and 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11644) Upgrade Hazelcast to 5.3.0
[ https://issues.apache.org/jira/browse/NIFI-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729456#comment-17729456 ] ASF subversion and git services commented on NIFI-11644: Commit 27f8a23b757f2132c10e28b29aa403b903221a1a in nifi's branch refs/heads/support/nifi-1.x from dependabot[bot] [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=27f8a23b75 ] NIFI-11644 Upgraded Hazelcast from 4.2.7 to 5.3.0 This closes #7282 Signed-off-by: David Handermann (cherry picked from commit 88d7b3905b7a5888daab29a8285f755defc7214b) > Upgrade Hazelcast to 5.3.0 > -- > > Key: NIFI-11644 > URL: https://issues.apache.org/jira/browse/NIFI-11644 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Time Spent: 20m > Remaining Estimate: 0h > > [Hazelcast 5.3.0|https://github.com/hazelcast/hazelcast/releases/tag/v5.3.0] > includes a number of new features and improvements while maintaining > compatibility with earlier releases. > The [Hazelcast Compatibility > Guarantees|https://docs.hazelcast.com/hazelcast/5.3/deploy/versioning-compatibility#compatibility-guarantees] > documentation notes that the Java client version 5 is compatible with both > Hazelcast 4 and 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11644) Upgrade Hazelcast to 5.3.0
[ https://issues.apache.org/jira/browse/NIFI-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729455#comment-17729455 ] ASF subversion and git services commented on NIFI-11644: Commit 88d7b3905b7a5888daab29a8285f755defc7214b in nifi's branch refs/heads/main from dependabot[bot] [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=88d7b3905b ] NIFI-11644 Upgraded Hazelcast from 4.2.7 to 5.3.0 This closes #7282 Signed-off-by: David Handermann > Upgrade Hazelcast to 5.3.0 > -- > > Key: NIFI-11644 > URL: https://issues.apache.org/jira/browse/NIFI-11644 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Time Spent: 20m > Remaining Estimate: 0h > > [Hazelcast 5.3.0|https://github.com/hazelcast/hazelcast/releases/tag/v5.3.0] > includes a number of new features and improvements while maintaining > compatibility with earlier releases. > The [Hazelcast Compatibility > Guarantees|https://docs.hazelcast.com/hazelcast/5.3/deploy/versioning-compatibility#compatibility-guarantees] > documentation notes that the Java client version 5 is compatible with both > Hazelcast 4 and 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #7282: NIFI-11644 Bump hazelcast from 4.2.7 to 5.3.0
exceptionfactory closed pull request #7282: NIFI-11644 Bump hazelcast from 4.2.7 to 5.3.0 URL: https://github.com/apache/nifi/pull/7282 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dependabot[bot] commented on pull request #7282: NIFI-11644 Bump hazelcast from 4.2.7 to 5.3.0
dependabot[bot] commented on PR #7282: URL: https://github.com/apache/nifi/pull/7282#issuecomment-1577379740 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11644) Upgrade Hazelcast to 5.3.0
David Handermann created NIFI-11644: --- Summary: Upgrade Hazelcast to 5.3.0 Key: NIFI-11644 URL: https://issues.apache.org/jira/browse/NIFI-11644 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: David Handermann Assignee: David Handermann [Hazelcast 5.3.0|https://github.com/hazelcast/hazelcast/releases/tag/v5.3.0] includes a number of new features and improvements while maintaining compatibility with earlier releases. The [Hazelcast Compatibility Guarantees|https://docs.hazelcast.com/hazelcast/5.3/deploy/versioning-compatibility#compatibility-guarantees] documentation notes that the Java client version 5 is compatible with both Hazelcast 4 and 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
exceptionfactory commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218506840
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
It should be possible to call `hasNext()` in any order. Taking a closer
look, it seems like the current implementation of `setCurrent` needs further
refinement because it sets `currentSheet` to `null` on every invocation. One
general approach could be to set the initial sheet and row in the constructor.
In general, it should be possible to call `hasNext()` and unlimited number
of times without advancing the iterator, and calling `next()` should advance
the iterator forward. When all sheets and rows have been read, `next()` should
throw an `NoSuchElementException` if called again.
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator she
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
exceptionfactory commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218506840
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
It should be possible to call `hasNext()` in any order. Taking a closer
look, it seems like the current implementation of `setCurrent` needs further
refinement because it sets `currentSheet` to `null` on every invocation. One
general approach could be to set the initial sheet and row in the constructor.
In general, it should be possible to call `hasNext()` and unlimited number
of times without advancing the iterator, and it calling `next()` should advance
the iterator forward. When all sheets and rows have been read, `next()` should
throw an `NoSuchElementException` if called again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
exceptionfactory commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218506840
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
It should be possible to call `hasNext()` in any order. Taking a closer
look, it seems like the current implementation of `setCurrent` needs further
refinement because it sets `currentSheet` to `null` on every invocation. One
general approach could be to set the initial sheet and row in the constructor.
In general, it should be possible to call `hasNext()` and unlimited number
of times without advancing the iterator, and it calling `next()` should advance
the iterator forward. When all sheets and rows have been read, `next()` should
throw an `IllegalStateException` if called again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
dan-s1 commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218495608
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
Just trying to understand. With that refactor then one could never call
hasNext before next since the instant variable currentRow is null. Should I
just use next and have hasNext be an unsupported operation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] r-sidd opened a new pull request, #7341: NIFI-11643: Upgrade geoip2 to 4.0.1
r-sidd opened a new pull request, #7341: URL: https://github.com/apache/nifi/pull/7341 # Summary [NIFI-11643](https://issues.apache.org/jira/browse/NIFI-11643) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
dan-s1 commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218495608
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
Just trying to understand. With that refactor then one could never call
hasNext before next since the instant variable currentRow is null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
dan-s1 commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218495608
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class RowIterator implements Iterator, Closeable {
+private final Workbook workbook;
+private final Iterator sheets;
+private final Map requiredSheets;
+private final int firstRow;
+private final ComponentLog logger;
+private Sheet currentSheet;
+private Iterator currentRows;
+private Row currentRow;
+
+public RowIterator(InputStream in, List requiredSheets, int
firstRow, ComponentLog logger) {
+this.workbook = StreamingReader.builder()
+.rowCacheSize(100)
+.bufferSize(4096)
+.open(in);
+this.sheets = this.workbook.iterator();
+this.requiredSheets = requiredSheets != null ? requiredSheets.stream()
+.collect(Collectors.toMap(key -> key, value -> Boolean.FALSE))
: new HashMap<>();
+this.firstRow = firstRow;
+this.logger = logger;
+}
+
+@Override
+public boolean hasNext() {
+setCurrent();
Review Comment:
Just trying to understand. With that refactor then one could never call
hasNext since the instant variable currentRow is null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-11643) Upgrade geoip2 to 4.0.1
Siddharth R created NIFI-11643: -- Summary: Upgrade geoip2 to 4.0.1 Key: NIFI-11643 URL: https://issues.apache.org/jira/browse/NIFI-11643 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.21.0, 2.0.0 Reporter: Siddharth R Assignee: Siddharth R Bump version from 2.16.1 to 4.0.1 to remediate findings: [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-42004] [https://mvnrepository.com/artifact/com.maxmind.geoip2/geoip2/2.16.1#:~:text=CVE%2D2022%2D42004-,CVE%2D2022%2D42003,-CVE%2D2021%2D46877] [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-46877] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dan-s1 commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
dan-s1 commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218460307
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/ExcelSchemaInference.java:
##
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import org.apache.nifi.schema.inference.FieldTypeInference;
+import org.apache.nifi.schema.inference.RecordSource;
+import org.apache.nifi.schema.inference.SchemaInferenceEngine;
+import org.apache.nifi.schema.inference.TimeValueInference;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.DataType;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.util.SchemaInferenceUtil;
+import org.apache.poi.ss.usermodel.Cell;
+import org.apache.poi.ss.usermodel.DataFormatter;
+import org.apache.poi.ss.usermodel.Row;
+
+import java.io.IOException;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+public class ExcelSchemaInference implements SchemaInferenceEngine {
+private final TimeValueInference timeValueInference;
+private final DataFormatter dataFormatter;
+
+public ExcelSchemaInference(TimeValueInference timeValueInference) {
+this(timeValueInference, null);
+}
+
+public ExcelSchemaInference(TimeValueInference timeValueInference, Locale
locale) {
+this.timeValueInference = timeValueInference;
+this.dataFormatter = locale == null ? new DataFormatter() : new
DataFormatter(locale);
+}
+
+@Override
+public RecordSchema inferSchema(RecordSource recordSource) throws
IOException {
+final Map typeMap = new LinkedHashMap<>();
+Row row;
+while((row = recordSource.next()) != null) {
+inferSchema(row, typeMap);
+}
+return createSchema(typeMap);
+}
+
+private void inferSchema(final Row row, final Map typeMap) {
+if (ExcelUtils.hasCells(row)) {
+IntStream.range(0, row.getLastCellNum())
+.forEach(index -> {
+final Cell cell = row.getCell(index);
+final String fieldName = Integer.toString(index);
Review Comment:
Sounds good to me. Per the comment on the query earlier this would certainly
be an improvement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-11642) Upgrade AWS SNS processors to use AWS SDK v2
Emilio Setiadarma created NIFI-11642: Summary: Upgrade AWS SNS processors to use AWS SDK v2 Key: NIFI-11642 URL: https://issues.apache.org/jira/browse/NIFI-11642 Project: Apache NiFi Issue Type: Improvement Reporter: Emilio Setiadarma Assignee: Emilio Setiadarma This issue will capture the effort involved in upgrading the AWS SNS Processors to use AWS v2 libraries -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] scottyaslan commented on pull request #7339: NIFI-11593: Fix hiding logic for properties
scottyaslan commented on PR #7339: URL: https://github.com/apache/nifi/pull/7339#issuecomment-1577248170 Reviewing... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11641) Upgrade Netty to 4.1.93
[ https://issues.apache.org/jira/browse/NIFI-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11641: Status: Patch Available (was: Open) > Upgrade Netty to 4.1.93 > --- > > Key: NIFI-11641 > URL: https://issues.apache.org/jira/browse/NIFI-11641 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 10m > Remaining Estimate: 0h > > Netty [4.1.93|https://netty.io/news/2023/05/25/4-1-93-Final.html] includes > several minor bug fixes and improvements, several related to HTTP/2 handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7340: NIFI-11641 Upgrade Netty from 4.1.92 to 4.1.93
exceptionfactory opened a new pull request, #7340: URL: https://github.com/apache/nifi/pull/7340 # Summary [NIFI-11641](https://issues.apache.org/jira/browse/NIFI-11641) Upgrades Netty from 4.1.92 to [4.1.93](https://netty.io/news/2023/05/25/4-1-93-Final.html), incorporating several minor bug fixes related to HTTP/2 and buffer handling. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11567) GeoEnrichIP processors should auto-reload the database file
[ https://issues.apache.org/jira/browse/NIFI-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729388#comment-17729388 ] ASF subversion and git services commented on NIFI-11567: Commit d200186096710096d566c871bf6dbc6e0de7484e in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d200186096 ] NIFI-11567: Auto-reload database file in GeoEnrichIP processors (#7266) NIFI-11567: Auto-reload database file in GeoEnrichIP processors > GeoEnrichIP processors should auto-reload the database file > --- > > Key: NIFI-11567 > URL: https://issues.apache.org/jira/browse/NIFI-11567 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the GeoEnrichIP processors only load the database when the > processor is scheduled. This requires a processor restart if the database > file changes. Instead, the processors should auto-reload the database file > when it detects a change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11567) GeoEnrichIP processors should auto-reload the database file
[ https://issues.apache.org/jira/browse/NIFI-11567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729389#comment-17729389 ] ASF subversion and git services commented on NIFI-11567: Commit 9949f079e1cb2acdc82ccdb80f03eba42d99253b in nifi's branch refs/heads/support/nifi-1.x from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9949f079e1 ] NIFI-11567: Auto-reload database file in GeoEnrichIP processors (#7266) NIFI-11567: Auto-reload database file in GeoEnrichIP processors > GeoEnrichIP processors should auto-reload the database file > --- > > Key: NIFI-11567 > URL: https://issues.apache.org/jira/browse/NIFI-11567 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.latest, 2.latest > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently the GeoEnrichIP processors only load the database when the > processor is scheduled. This requires a processor restart if the database > file changes. Instead, the processors should auto-reload the database file > when it detects a change. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11641) Upgrade Netty to 4.1.93
David Handermann created NIFI-11641: --- Summary: Upgrade Netty to 4.1.93 Key: NIFI-11641 URL: https://issues.apache.org/jira/browse/NIFI-11641 Project: Apache NiFi Issue Type: Task Components: Extensions Reporter: David Handermann Assignee: David Handermann Fix For: 1.latest, 2.latest Netty [4.1.93|https://netty.io/news/2023/05/25/4-1-93-Final.html] includes several minor bug fixes and improvements, several related to HTTP/2 handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mtien-apache commented on pull request #7339: NIFI-11593: Fix hiding logic for properties
mtien-apache commented on PR #7339: URL: https://github.com/apache/nifi/pull/7339#issuecomment-1577218837 Reviewing now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] sardell opened a new pull request, #7339: NIFI-11593: Fix hiding logic for properties
sardell opened a new pull request, #7339: URL: https://github.com/apache/nifi/pull/7339 # Summary [NIFI-11593](https://issues.apache.org/jira/browse/NIFI-11593) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-11640) Update download-maven-plugin to 1.6.8 for Registry
[ https://issues.apache.org/jira/browse/NIFI-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-11640. - Fix Version/s: 2.0.0 1.22.0 (was: 1.latest) (was: 2.latest) Resolution: Fixed > Update download-maven-plugin to 1.6.8 for Registry > -- > > Key: NIFI-11640 > URL: https://issues.apache.org/jira/browse/NIFI-11640 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry, Tools and Build >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0, 1.22.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Bump download-maven-plugin to 1.7.0 to remediate multiple CVEs: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-29425] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13956] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1002200] > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11640) Update download-maven-plugin to 1.6.8 for Registry
[ https://issues.apache.org/jira/browse/NIFI-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729378#comment-17729378 ] ASF subversion and git services commented on NIFI-11640: Commit dfeb4462302c59f2be0b68e9e8080649605f2108 in nifi's branch refs/heads/support/nifi-1.x from siddr [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=dfeb446230 ] NIFI-11640 Upgraded download-maven-plugin from 1.2.1 to 1.6.8 This closes #7337 Signed-off-by: David Handermann (cherry picked from commit 0cd511f1aa1074e002fad8e96f8ca25971db) > Update download-maven-plugin to 1.6.8 for Registry > -- > > Key: NIFI-11640 > URL: https://issues.apache.org/jira/browse/NIFI-11640 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry, Tools and Build >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 50m > Remaining Estimate: 0h > > Bump download-maven-plugin to 1.7.0 to remediate multiple CVEs: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-29425] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13956] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1002200] > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11640) Update download-maven-plugin to 1.6.8 for Registry
[ https://issues.apache.org/jira/browse/NIFI-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729376#comment-17729376 ] ASF subversion and git services commented on NIFI-11640: Commit 0cd511f1aa1074e002fad8e96f8ca25971db in nifi's branch refs/heads/main from siddr [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0cd511f1aa ] NIFI-11640 Upgraded download-maven-plugin from 1.2.1 to 1.6.8 This closes #7337 Signed-off-by: David Handermann > Update download-maven-plugin to 1.6.8 for Registry > -- > > Key: NIFI-11640 > URL: https://issues.apache.org/jira/browse/NIFI-11640 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry, Tools and Build >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 50m > Remaining Estimate: 0h > > Bump download-maven-plugin to 1.7.0 to remediate multiple CVEs: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-29425] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13956] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1002200] > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11640) Update download-maven-plugin to 1.6.8 for Registry
[ https://issues.apache.org/jira/browse/NIFI-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11640: Component/s: NiFi Registry Tools and Build > Update download-maven-plugin to 1.6.8 for Registry > -- > > Key: NIFI-11640 > URL: https://issues.apache.org/jira/browse/NIFI-11640 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry, Tools and Build >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 40m > Remaining Estimate: 0h > > Bump download-maven-plugin to 1.7.0 to remediate multiple CVEs: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-29425] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13956] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1002200] > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11631) Add Oracle support for NiFi Registry
[ https://issues.apache.org/jira/browse/NIFI-11631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-11631: Summary: Add Oracle support for NiFi Registry (was: Add OracleDB support for Nifi Registry) > Add Oracle support for NiFi Registry > > > Key: NIFI-11631 > URL: https://issues.apache.org/jira/browse/NIFI-11631 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Registry >Reporter: Kalmár Róbert >Assignee: Kalmár Róbert >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11640) Update download-maven-plugin to 1.6.8 for Registry
[ https://issues.apache.org/jira/browse/NIFI-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11640: Summary: Update download-maven-plugin to 1.6.8 for Registry (was: Update download-maven-plugin to 1.7.0) > Update download-maven-plugin to 1.6.8 for Registry > -- > > Key: NIFI-11640 > URL: https://issues.apache.org/jira/browse/NIFI-11640 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 40m > Remaining Estimate: 0h > > Bump download-maven-plugin to 1.7.0 to remediate multiple CVEs: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-29425] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13956] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1002200] > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mattyb149 commented on pull request #7338: NIFI-11631 Add OracleDB support for Nifi Registry
mattyb149 commented on PR #7338: URL: https://github.com/apache/nifi/pull/7338#issuecomment-1577116427 Thanks for this! I tried this at one point ([oracle_registry](https://github.com/mattyb149/nifi/commit/e912b9155cce0b919f0a3d4d9edfc55c7038a65b)) but had trouble setting up integration tests using testcontainers. I think we really need integration tests here like we have with the other implementation(s). Feel free to compare your PR against my aforementioned commit to make sure we are on the same page. I tested mine manually and everything worked fine, but I never got the integration test stuff working. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
exceptionfactory commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1218310754
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/ExcelSchemaInference.java:
##
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import org.apache.nifi.schema.inference.FieldTypeInference;
+import org.apache.nifi.schema.inference.RecordSource;
+import org.apache.nifi.schema.inference.SchemaInferenceEngine;
+import org.apache.nifi.schema.inference.TimeValueInference;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.DataType;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.util.SchemaInferenceUtil;
+import org.apache.poi.ss.usermodel.Cell;
+import org.apache.poi.ss.usermodel.DataFormatter;
+import org.apache.poi.ss.usermodel.Row;
+
+import java.io.IOException;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+public class ExcelSchemaInference implements SchemaInferenceEngine {
+private final TimeValueInference timeValueInference;
+private final DataFormatter dataFormatter;
+
+public ExcelSchemaInference(TimeValueInference timeValueInference) {
+this(timeValueInference, null);
+}
+
+public ExcelSchemaInference(TimeValueInference timeValueInference, Locale
locale) {
+this.timeValueInference = timeValueInference;
+this.dataFormatter = locale == null ? new DataFormatter() : new
DataFormatter(locale);
+}
+
+@Override
+public RecordSchema inferSchema(RecordSource recordSource) throws
IOException {
+final Map typeMap = new LinkedHashMap<>();
+Row row;
+while((row = recordSource.next()) != null) {
+inferSchema(row, typeMap);
+}
+return createSchema(typeMap);
+}
+
+private void inferSchema(final Row row, final Map typeMap) {
+if (ExcelUtils.hasCells(row)) {
+IntStream.range(0, row.getLastCellNum())
+.forEach(index -> {
+final Cell cell = row.getCell(index);
+final String fieldName = Integer.toString(index);
Review Comment:
What do you think of prefixing this field name with `column_`? So the first
field name would be `column_0`? That would be easier to work with in absence of
a defined schema.
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/RowIterator.java:
##
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import com.github.pjfanning.xlsx.StreamingReader;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.poi.ss.usermodel.Row;
+import org.apache.poi.ss.usermodel.Sheet;
+import org.apache.poi.ss.usermodel.Workbook;
+
+import java.io.Closeable;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+impor
[jira] [Updated] (NIFI-11593) UI - Dependent Properties with No Allowable Values
[ https://issues.apache.org/jira/browse/NIFI-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11593: Fix Version/s: 2.0.0 1.22.0 (was: 1.latest) (was: 2.latest) > UI - Dependent Properties with No Allowable Values > -- > > Key: NIFI-11593 > URL: https://issues.apache.org/jira/browse/NIFI-11593 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Shane Ardell >Priority: Major > Fix For: 2.0.0, 1.22.0 > > > The front end does not support dependent properties correctly when the > underlying Property does not have allowable values. When the dependent > property does not have allowable values, any properties that depend on it > should be shown when it is set (contains any value). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11593) UI - Dependent Properties with No Allowable Values
[ https://issues.apache.org/jira/browse/NIFI-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11593: Fix Version/s: (was: 2.0.0) (was: 1.22.0) > UI - Dependent Properties with No Allowable Values > -- > > Key: NIFI-11593 > URL: https://issues.apache.org/jira/browse/NIFI-11593 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Shane Ardell >Priority: Major > > The front end does not support dependent properties correctly when the > underlying Property does not have allowable values. When the dependent > property does not have allowable values, any properties that depend on it > should be shown when it is set (contains any value). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11593) UI - Dependent Properties with No Allowable Values
[ https://issues.apache.org/jira/browse/NIFI-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729372#comment-17729372 ] Joe Witt commented on NIFI-11593: - going to wait for this to be fixed since this is a regression since 1.21 and it could cause properties to be not visible that should be > UI - Dependent Properties with No Allowable Values > -- > > Key: NIFI-11593 > URL: https://issues.apache.org/jira/browse/NIFI-11593 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Shane Ardell >Priority: Major > Fix For: 2.0.0, 1.22.0 > > > The front end does not support dependent properties correctly when the > underlying Property does not have allowable values. When the dependent > property does not have allowable values, any properties that depend on it > should be shown when it is set (contains any value). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11593) UI - Dependent Properties with No Allowable Values
[ https://issues.apache.org/jira/browse/NIFI-11593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11593: Fix Version/s: 1.latest 2.latest > UI - Dependent Properties with No Allowable Values > -- > > Key: NIFI-11593 > URL: https://issues.apache.org/jira/browse/NIFI-11593 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Shane Ardell >Priority: Major > Fix For: 1.latest, 2.latest > > > The front end does not support dependent properties correctly when the > underlying Property does not have allowable values. When the dependent > property does not have allowable values, any properties that depend on it > should be shown when it is set (contains any value). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11640) Update download-maven-plugin to 1.7.0
[ https://issues.apache.org/jira/browse/NIFI-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11640: Fix Version/s: 1.latest 2.latest > Update download-maven-plugin to 1.7.0 > - > > Key: NIFI-11640 > URL: https://issues.apache.org/jira/browse/NIFI-11640 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.latest, 2.latest > > Time Spent: 0.5h > Remaining Estimate: 0h > > Bump download-maven-plugin to 1.7.0 to remediate multiple CVEs: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-29425] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13956] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1002200] > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11640) Update download-maven-plugin to 1.7.0
[ https://issues.apache.org/jira/browse/NIFI-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-11640: Fix Version/s: (was: 2.0.0) (was: 1.22.0) > Update download-maven-plugin to 1.7.0 > - > > Key: NIFI-11640 > URL: https://issues.apache.org/jira/browse/NIFI-11640 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Time Spent: 0.5h > Remaining Estimate: 0h > > Bump download-maven-plugin to 1.7.0 to remediate multiple CVEs: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-29425] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13956] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1002200] > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11640) Update download-maven-plugin to 1.7.0
[ https://issues.apache.org/jira/browse/NIFI-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729368#comment-17729368 ] Joe Witt commented on NIFI-11640: - Fix versions can be applied once there is a reviewer that looks into this and is merging. Thanks > Update download-maven-plugin to 1.7.0 > - > > Key: NIFI-11640 > URL: https://issues.apache.org/jira/browse/NIFI-11640 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Siddharth R >Assignee: Siddharth R >Priority: Minor > Labels: dependency-upgrade > Time Spent: 0.5h > Remaining Estimate: 0h > > Bump download-maven-plugin to 1.7.0 to remediate multiple CVEs: > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-29425] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-13956] > [https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1002200] > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11530) Disk full even with nifi.content.repository.archive.max.usage.percentage set to 50%
[ https://issues.apache.org/jira/browse/NIFI-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729366#comment-17729366 ] Joe Witt commented on NIFI-11530: - Hello All of the details that matter for this will come from how it is configured. It appears you are using a single shared disk on each node whereby that disk has content, flowfile, and provenance repositories. While this is possible it is not recommended. It being possible means you will need to have specific configurations that work for it as well. What are your settings for content storage? Provenance storage? And flowfile storage? Those are all in nifi.properties. So far looking at this I see no sign of a bug. Just a sign of a configuration that won't likely work well or as you likely intend. > Disk full even with nifi.content.repository.archive.max.usage.percentage set > to 50% > --- > > Key: NIFI-11530 > URL: https://issues.apache.org/jira/browse/NIFI-11530 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.20.0 > Environment: Ubuntu 20.04.5 LTS >Reporter: Giovanni >Priority: Major > Attachments: 20230605_disk_usage.jpg, content_archive.jpg, > flowfile_archive.jpg, jvm.jpg, nifi1-app.log, nifi2-app.log, nifi3-app.log, > nifi_bug.jpg, provenance_archive.jpg > > > Nifi primary node reports disk full causing all nodes to stop working. > Restarting nifi service does not resolve. > Restarting the VM does not resolve. > The only way to fix is to clean te content_repository dir: > rm -rf ./nifi/content_repository/* > > Unfortunately I have no logs of the issue ongoing. > > UPDATE: > I'm having the problem again. > Every archive size is more than 50% on each node, with 70%+ peak on > coordinator node (see attachments). > I'm also attaching nifi-app.log this time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11630) Deprecate ECMAScript Script Engine
[
https://issues.apache.org/jira/browse/NIFI-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-11630:
Fix Version/s: 1.22.0
(was: 1.latest)
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Deprecate ECMAScript Script Engine
> --
>
> Key: NIFI-11630
> URL: https://issues.apache.org/jira/browse/NIFI-11630
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 1.22.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The {{ECMAScript}} Script Engine for multiple scripted Processors and
> Controller Services should be deprecated for removal in NiFi 2.0.
> The {{ECMAScript}} engine supports JavaScript-compatible scripted components.
> The Nashorn engine was deprecated in Java 11 as described in [JEP
> 335|https://openjdk.org/jeps/335] and is no longer available in Java 17.
> Alternative JavaScript engines could be considered separately, but
> {{ECMAScript}} should be deprecated for removal.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11630) Deprecate ECMAScript Script Engine
[
https://issues.apache.org/jira/browse/NIFI-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729365#comment-17729365
]
ASF subversion and git services commented on NIFI-11630:
Commit 7d057e9ef539741df88fd99cf9243bd9ff670876 in nifi's branch
refs/heads/support/nifi-1.x from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7d057e9ef5 ]
NIFI-11630 Deprecated ECMAScript Script Engine
Signed-off-by: Matt Burgess
This closes #7329
> Deprecate ECMAScript Script Engine
> --
>
> Key: NIFI-11630
> URL: https://issues.apache.org/jira/browse/NIFI-11630
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 1.latest
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The {{ECMAScript}} Script Engine for multiple scripted Processors and
> Controller Services should be deprecated for removal in NiFi 2.0.
> The {{ECMAScript}} engine supports JavaScript-compatible scripted components.
> The Nashorn engine was deprecated in Java 11 as described in [JEP
> 335|https://openjdk.org/jeps/335] and is no longer available in Java 17.
> Alternative JavaScript engines could be considered separately, but
> {{ECMAScript}} should be deprecated for removal.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] mattyb149 closed pull request #7329: NIFI-11630 Deprecate ECMAScript Script Engine
mattyb149 closed pull request #7329: NIFI-11630 Deprecate ECMAScript Script Engine URL: https://github.com/apache/nifi/pull/7329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #7329: NIFI-11630 Deprecate ECMAScript Script Engine
mattyb149 commented on PR #7329: URL: https://github.com/apache/nifi/pull/7329#issuecomment-1577066158 +1 LGTM, merging to support/nifi-1.x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11530) Disk full even with nifi.content.repository.archive.max.usage.percentage set to 50%
[ https://issues.apache.org/jira/browse/NIFI-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni updated NIFI-11530: Description: Nifi primary node reports disk full causing all nodes to stop working. Restarting nifi service does not resolve. Restarting the VM does not resolve. The only way to fix is to clean te content_repository dir: rm -rf ./nifi/content_repository/* Unfortunately I have no logs of the issue ongoing. UPDATE: I'm having the problem again. Every archive size is more than 50% on each node, with 70%+ peak on coordinator node (see attachments). I'm also attaching nifi-app.log this time. was: Nifi primary node reports disk full causing all nodes to stop working. Restarting nifi service does not resolve. Restarting the VM does not resolve. The only way to fix is to clean te content_repository dir: rm -rf ./nifi/content_repository/* Unfortunately I have no logs of the issue ongoing. UPDATE: I'm having the problem again. Every archive size is more than 50% on each node, with 70%+ peak on coordinator node (see attachments). I'm also attaching nifi-app.log this time. > Disk full even with nifi.content.repository.archive.max.usage.percentage set > to 50% > --- > > Key: NIFI-11530 > URL: https://issues.apache.org/jira/browse/NIFI-11530 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.20.0 > Environment: Ubuntu 20.04.5 LTS >Reporter: Giovanni >Priority: Major > Attachments: 20230605_disk_usage.jpg, content_archive.jpg, > flowfile_archive.jpg, jvm.jpg, nifi1-app.log, nifi2-app.log, nifi3-app.log, > nifi_bug.jpg, provenance_archive.jpg > > > Nifi primary node reports disk full causing all nodes to stop working. > Restarting nifi service does not resolve. > Restarting the VM does not resolve. > The only way to fix is to clean te content_repository dir: > rm -rf ./nifi/content_repository/* > > Unfortunately I have no logs of the issue ongoing. > > UPDATE: > I'm having the problem again. > Every archive size is more than 50% on each node, with 70%+ peak on > coordinator node (see attachments). > I'm also attaching nifi-app.log this time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11530) Disk full even with nifi.content.repository.archive.max.usage.percentage set to 50%
[ https://issues.apache.org/jira/browse/NIFI-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni updated NIFI-11530: Description: Nifi primary node reports disk full causing all nodes to stop working. Restarting nifi service does not resolve. Restarting the VM does not resolve. The only way to fix is to clean te content_repository dir: rm -rf ./nifi/content_repository/* Unfortunately I have no logs of the issue ongoing. UPDATE: I'm having the problem again. Every archive size is more than 50% on each node, with 70%+ peak on coordinator node (see attachments). I'm also attaching nifi-app.log this time. was: Nifi primary node reports disk full causing all nodes to stop working. Restarting nifi service does not resolve. Restarting the VM does not resolve. The only way to fix is to clean te content_repository dir: rm -rf ./nifi/content_repository/* Unfortunately I have no logs of the issue ongoing. UPDATE: I'm having the problem again. Every archive size is more than 50% on each node, with 70%+ peak on coordinator node (see attachments). I'm also attaching nifi-app.log that is reporting the following error: > Disk full even with nifi.content.repository.archive.max.usage.percentage set > to 50% > --- > > Key: NIFI-11530 > URL: https://issues.apache.org/jira/browse/NIFI-11530 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.20.0 > Environment: Ubuntu 20.04.5 LTS >Reporter: Giovanni >Priority: Major > Attachments: 20230605_disk_usage.jpg, content_archive.jpg, > flowfile_archive.jpg, jvm.jpg, nifi1-app.log, nifi2-app.log, nifi3-app.log, > nifi_bug.jpg, provenance_archive.jpg > > > Nifi primary node reports disk full causing all nodes to stop working. > Restarting nifi service does not resolve. > Restarting the VM does not resolve. > The only way to fix is to clean te content_repository dir: > rm -rf ./nifi/content_repository/* > > Unfortunately I have no logs of the issue ongoing. > UPDATE: > I'm having the problem again. > Every archive size is more than 50% on each node, with 70%+ peak on > coordinator node (see attachments). > I'm also attaching nifi-app.log this time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11530) Disk full even with nifi.content.repository.archive.max.usage.percentage set to 50%
[ https://issues.apache.org/jira/browse/NIFI-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni updated NIFI-11530: Attachment: 20230605_disk_usage.jpg > Disk full even with nifi.content.repository.archive.max.usage.percentage set > to 50% > --- > > Key: NIFI-11530 > URL: https://issues.apache.org/jira/browse/NIFI-11530 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.20.0 > Environment: Ubuntu 20.04.5 LTS >Reporter: Giovanni >Priority: Major > Attachments: 20230605_disk_usage.jpg, content_archive.jpg, > flowfile_archive.jpg, jvm.jpg, nifi1-app.log, nifi2-app.log, nifi3-app.log, > nifi_bug.jpg, provenance_archive.jpg > > > Nifi primary node reports disk full causing all nodes to stop working. > Restarting nifi service does not resolve. > Restarting the VM does not resolve. > The only way to fix is to clean te content_repository dir: > rm -rf ./nifi/content_repository/* > > Unfortunately I have no logs of the issue ongoing. > UPDATE: > I'm having the problem again. > Every archive size is more than 50% on each node, with 70%+ peak on > coordinator node (see attachments). > I'm also attaching nifi-app.log that is reporting the following error: -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11530) Disk full even with nifi.content.repository.archive.max.usage.percentage set to 50%
[ https://issues.apache.org/jira/browse/NIFI-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni updated NIFI-11530: Description: Nifi primary node reports disk full causing all nodes to stop working. Restarting nifi service does not resolve. Restarting the VM does not resolve. The only way to fix is to clean te content_repository dir: rm -rf ./nifi/content_repository/* Unfortunately I have no logs of the issue ongoing. UPDATE: I'm having the problem again. Every archive size is more than 50% on each node, with 70%+ peak on coordinator node (see attachments). I'm also attaching nifi-app.log that is reporting the following error: was: Nifi primary node reports disk full causing all nodes to stop working. Restarting nifi service does not resolve. Restarting the VM does not resolve. The only way to fix is to clean te content_repository dir: rm -rf ./nifi/content_repository/* Unfortunately I have no logs of the issue ongoing. > Disk full even with nifi.content.repository.archive.max.usage.percentage set > to 50% > --- > > Key: NIFI-11530 > URL: https://issues.apache.org/jira/browse/NIFI-11530 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.20.0 > Environment: Ubuntu 20.04.5 LTS >Reporter: Giovanni >Priority: Major > Attachments: content_archive.jpg, flowfile_archive.jpg, jvm.jpg, > nifi1-app.log, nifi2-app.log, nifi3-app.log, nifi_bug.jpg, > provenance_archive.jpg > > > Nifi primary node reports disk full causing all nodes to stop working. > Restarting nifi service does not resolve. > Restarting the VM does not resolve. > The only way to fix is to clean te content_repository dir: > rm -rf ./nifi/content_repository/* > > Unfortunately I have no logs of the issue ongoing. > UPDATE: > I'm having the problem again. > Every archive size is more than 50% on each node, with 70%+ peak on > coordinator node (see attachments). > I'm also attaching nifi-app.log that is reporting the following error: -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11530) Disk full even with nifi.content.repository.archive.max.usage.percentage set to 50%
[ https://issues.apache.org/jira/browse/NIFI-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni updated NIFI-11530: Attachment: nifi1-app.log nifi2-app.log nifi3-app.log > Disk full even with nifi.content.repository.archive.max.usage.percentage set > to 50% > --- > > Key: NIFI-11530 > URL: https://issues.apache.org/jira/browse/NIFI-11530 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.20.0 > Environment: Ubuntu 20.04.5 LTS >Reporter: Giovanni >Priority: Major > Attachments: content_archive.jpg, flowfile_archive.jpg, jvm.jpg, > nifi1-app.log, nifi2-app.log, nifi3-app.log, nifi_bug.jpg, > provenance_archive.jpg > > > Nifi primary node reports disk full causing all nodes to stop working. > Restarting nifi service does not resolve. > Restarting the VM does not resolve. > The only way to fix is to clean te content_repository dir: > rm -rf ./nifi/content_repository/* > > Unfortunately I have no logs of the issue ongoing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[
https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Bende updated NIFI-11464:
---
Fix Version/s: 2.0.0
1.22.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> importing a versioned flow with a nested versioned flow shows nested
> versioned flow with invalid controller services.
> -
>
> Key: NIFI-11464
> URL: https://issues.apache.org/jira/browse/NIFI-11464
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.21.0
> Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
>Reporter: Patrick A. Mol
>Assignee: Bryan Bende
>Priority: Major
> Fix For: 2.0.0, 1.22.0
>
> Attachments: exported_flow_versions_pretty.zip,
> image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png,
> nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When a flow (reusable_flow_Q) has controller services inherited from the
> hierarchy (process group reusables) and a version of the flow is stored, the
> flow version contains the references to these external controller services
> (as seen in an exported flow version [see below]).
> When this versioned flow is imported in another flow (tenant_flows) the
> controller services need to be reset to the controller services in the new
> hierarchy.
> When we have a working flow with the nested versioned flow ready in
> development we can check this flow into version control.
> When we then deploy the flow in production, the nested versioned flow shows
> up with invalid components. It shows the external controller service
> identifiers as stored in the flow version.
> When we then go back to development version of tenant_flows and make a minor
> change to the nested versioned flow reusable_flow_Q and commit this change to
> version control.
> Due to this version change, we need to also commit the changes for the
> tenant_flows process group.
> When we now go back to production, and import this new version of
> tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid
> controller services.
> If you have several flows under development using the same reusable
> components,
> you will likely end up with invalid components after import.
> Depending on the amount of versioned flows used, it could be a lot of work.
> It could also lead to issues when using the ExecuteStateless processor.
> Please see attached template nested_version_flow_issue.xml for a starting
> point to reproduce the issue. It contains the steps.
> Screenprints are attached in a zip file show the process and diagnosis.
> Controller services identifiers in version 2.
> {code:java}
> $ fgrep -C 4 reusables_avro reusable_flow_Q.json.pretty
> "controllerServices": [
> {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "instanceIdentifier": "8f647d06-0187-1000-4be9-14a61f55d904",
> "name": "reusables_avro_reader",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroReader",
> "bundle": {
> "group": "org.apache.nifi",
> --
> },
> {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "instanceIdentifier": "8f64f2c6-0187-1000-7557-ca63c88054dd",
> "name": "reusables_avro_writer",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroRecordSetWriter",
> "bundle": {
> "group": "org.apache.nifi",
> $ head -15 reusable_flow_Q-version-2.json.pretty
> {
> "externalControllerServices": {
> "dc884171-4d75-3854-8604-afab91bd0e60": {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "name": "reusables_avro_reader"
> },
> "b512b238-cdee-3642-b5cb-0c98d30dd133": {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "name": "reusables_avro_writer"
> }
> },
> "flowContents": {
> "comments": "used to perform Q ...",
> "componentType": "PROCESS_GROUP",
> "connections": [
> {code}
> Controller services identifiers with version 3 committed in process group
> "tenant_flows".
> {code:java}
> pmo@hpmo:~/Documents.local/nested_versioned_flows_controller_issue$ fgrep -C
> 4 tenant_flow_avro tenant_flows-version-1.json.pretty
> ],
> "groupIdentifier": "a984831b-8587-3e17-bbbc-ef4b85c3898d",
> "identifier": "5d9df37d-2a52-3f6e-8cd3-3d3ea9550d22",
> "instanceIdentifier": "8f6cb319-0187-1000-b7fa-83340f7055f7",
> "name": "tenant_flow_avro_writer",
> "properties": {
> "compression-format": "NONE",
>
[jira] [Commented] (NIFI-11464) importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.
[
https://issues.apache.org/jira/browse/NIFI-11464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729343#comment-17729343
]
ASF subversion and git services commented on NIFI-11464:
Commit 64b110c2929b9d9de4d354d0ecc05c8b9c0fb8dc in nifi's branch
refs/heads/support/nifi-1.x from Bryan Bende
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=64b110c292 ]
NIFI-11464 Improvements for importing nested versioned flows (#7218)
* NIFI-11464 Improvements for importing nested versioned flows
- Introduce FlowSnapshotContainer to return root snapshot + children
- Introduce ControllerServiceResolver to extract logic from service facade
- Update resolution logic to correctly consider all services in the hierarchy
- Merge additional parameter contexts and parameter providers from child to
parent
- Add unit test for controller service resolver
- Replace use of emptSet/emptyMap with new set/map instance
> importing a versioned flow with a nested versioned flow shows nested
> versioned flow with invalid controller services.
> -
>
> Key: NIFI-11464
> URL: https://issues.apache.org/jira/browse/NIFI-11464
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.21.0
> Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
>Reporter: Patrick A. Mol
>Assignee: Bryan Bende
>Priority: Major
> Attachments: exported_flow_versions_pretty.zip,
> image-2023-04-17-17-34-08-898.png, image-2023-04-17-18-06-16-102.png,
> nested_versioned_flow_issue.xml, screenprints_reproduction_steps.zip
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When a flow (reusable_flow_Q) has controller services inherited from the
> hierarchy (process group reusables) and a version of the flow is stored, the
> flow version contains the references to these external controller services
> (as seen in an exported flow version [see below]).
> When this versioned flow is imported in another flow (tenant_flows) the
> controller services need to be reset to the controller services in the new
> hierarchy.
> When we have a working flow with the nested versioned flow ready in
> development we can check this flow into version control.
> When we then deploy the flow in production, the nested versioned flow shows
> up with invalid components. It shows the external controller service
> identifiers as stored in the flow version.
> When we then go back to development version of tenant_flows and make a minor
> change to the nested versioned flow reusable_flow_Q and commit this change to
> version control.
> Due to this version change, we need to also commit the changes for the
> tenant_flows process group.
> When we now go back to production, and import this new version of
> tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid
> controller services.
> If you have several flows under development using the same reusable
> components,
> you will likely end up with invalid components after import.
> Depending on the amount of versioned flows used, it could be a lot of work.
> It could also lead to issues when using the ExecuteStateless processor.
> Please see attached template nested_version_flow_issue.xml for a starting
> point to reproduce the issue. It contains the steps.
> Screenprints are attached in a zip file show the process and diagnosis.
> Controller services identifiers in version 2.
> {code:java}
> $ fgrep -C 4 reusables_avro reusable_flow_Q.json.pretty
> "controllerServices": [
> {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "instanceIdentifier": "8f647d06-0187-1000-4be9-14a61f55d904",
> "name": "reusables_avro_reader",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroReader",
> "bundle": {
> "group": "org.apache.nifi",
> --
> },
> {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "instanceIdentifier": "8f64f2c6-0187-1000-7557-ca63c88054dd",
> "name": "reusables_avro_writer",
> "comments": "",
> "type": "org.apache.nifi.avro.AvroRecordSetWriter",
> "bundle": {
> "group": "org.apache.nifi",
> $ head -15 reusable_flow_Q-version-2.json.pretty
> {
> "externalControllerServices": {
> "dc884171-4d75-3854-8604-afab91bd0e60": {
> "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
> "name": "reusables_avro_reader"
> },
> "b512b238-cdee-3642-b5cb-0c98d30dd133": {
> "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
> "name": "reusables_avro_writer"
> }
> },
> "flowContents": {
> "comments": "used to perform Q ..."
