[
https://issues.apache.org/jira/browse/NIFI-8423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325735#comment-17325735
]
Josef Zahner edited comment on NIFI-8423 at 4/20/21, 12:03 PM:
---------------------------------------------------------------
I don't know whether p-li-nifi-05 is the only node which shows the discrepancy,
under normal circumstances no ERRORs will be generated so I don't know when and
if it occurs on other nodes. I've already restarted NiFi on all nodes multiple
times. It's really difficult to tell if it's always the same node (or multiple
nodes)
What I can tell is that NiFi runs only once:
{code:java}
[user@p-li-nifi-05 ~]$ ps aux | grep nifi
user 19597 0.0 0.0 112808 968 pts/0 S+ 13:13 0:00 grep
--color=auto nifi
nifi 51798 0.0 0.0 113412 760 ? S 10:46 0:00 /bin/sh
/opt/nifi/bin/nifi.sh start
nifi 51800 0.0 0.0 7093000 197364 ? Sl 10:46 0:06
/usr/lib/jvm/java/bin/java -cp
/opt/nifi-1.13.2/conf:/opt/nifi-1.13.2/lib/bootstrap/* -Xms12m -Xmx24m
-Dorg.apache.nifi.bootstrap.config.log.dir=/var/log/nifi
-Dorg.apache.nifi.bootstrap.config.pid.dir=/var/run/nifi
-Dorg.apache.nifi.bootstrap.config.file=/opt/nifi-1.13.2/conf/bootstrap.conf
org.apache.nifi.bootstrap.RunNiFi start
nifi 51872 332 14.0 74496404 37058284 ? Sl 10:47 487:17
/usr/lib/jvm/java/bin/java -classpath
/opt/nifi-1.13.2/./conf:/opt/nifi-1.13.2/./lib/javax.servlet-api-3.1.0.jar:/opt/nifi-1.13.2/./lib/jetty-schemas-3.1.jar:/opt/nifi-1.13.2/./lib/logback-classic-1.2.3.jar:/opt/nifi-1.13.2/./lib/logback-core-1.2.3.jar:/opt/nifi-1.13.2/./lib/jcl-over-slf4j-1.7.30.jar:/opt/nifi-1.13.2/./lib/jul-to-slf4j-1.7.30.jar:/opt/nifi-1.13.2/./lib/log4j-over-slf4j-1.7.30.jar:/opt/nifi-1.13.2/./lib/slf4j-api-1.7.30.jar:/opt/nifi-1.13.2/./lib/nifi-api-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-framework-api-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-server-api-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-runtime-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-nar-utils-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-properties-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-stateless-bootstrap-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-stateless-api-1.13.2.jar
-XX:+PrintGCDetails -Dcom.sun.management.jmxremote.rmi.port=30008
-XX:+UseGCLogFileRotation -Xloggc:/var/log/nifi/gc.log
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=30008
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=true -Duser.timezone=Europe/Zurich
-Dorg.apache.jasper.compiler.disablejsr199=true -Xmx31g -Xms8g
-Dcurator-log-only-first-connection-issue-as-error-level=true
-Djavax.security.auth.useSubjectCredsOnly=true
-Djava.security.egd=file:/dev/urandom -Dzookeeper.admin.enableServer=false
-Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true -XX:ReservedCodeCacheSize=256m -XX:+UseG1GC
-Djava.protocol.handler.pkgs=sun.net.www.protocol -XX:NumberOfGCLogFiles=10
-XX:+UseCodeCacheFlushing -XX:CodeCacheMinimumFreeSpace=10m
-XX:GCLogFileSize=10m
-Dnifi.properties.file.path=/opt/nifi-1.13.2/./conf/nifi.properties
-Dnifi.bootstrap.listen.port=39853 -Dapp=NiFi
-Dorg.apache.nifi.bootstrap.config.log.dir=/var/log/nifi org.apache.nifi.NiFi
nifi 53618 0.3 0.0 31964 23484 ? Ssl 09:11 0:44
/opt/pushgateway/pushgateway --web.listen-address 0.0.0.0:9091
{code}
We have a systemd services for NiFi, so in theory it should never start more
than one NiFi:
{code:java}
[user@p-li-nifi-05 ~]$ cat /etc/systemd/system/nifi.service
[Unit]
Description=Apache NiFi
After=network.target[Service]
Type=forking
User=nifi
Group=nifi
RuntimeDirectory=nifi
RuntimeDirectoryMode=0775
ExecStart=/opt/nifi/bin/nifi.sh start
ExecStop=/opt/nifi/bin/nifi.sh stop
ExecReload=/opt/nifi/bin/nifi.sh restart
LimitNOFILE=1000000[Install]
WantedBy=multi-user.target
{code}
was (Author: jzahner):
I don't know whether p-li-nifi-05 is the only node which shows the discrepancy,
under normal circumstances no ERRORs will be generated so I don't know when and
if it occurs on other nodes. I've already restarted NiFi on all nodes multiple
times. It's really difficult to tell if it's always the same node (or multiple
nodes)
What I can tell is that NiFi runs only once:
{code:java}
[user@p-li-nifi-05 ~]$ ps aux | grep nifi
user 19597 0.0 0.0 112808 968 pts/0 S+ 13:13 0:00 grep
--color=auto nifi
nifi 51798 0.0 0.0 113412 760 ? S 10:46 0:00 /bin/sh
/opt/nifi/bin/nifi.sh start
nifi 51800 0.0 0.0 7093000 197364 ? Sl 10:46 0:06
/usr/lib/jvm/java/bin/java -cp
/opt/nifi-1.13.2/conf:/opt/nifi-1.13.2/lib/bootstrap/* -Xms12m -Xmx24m
-Dorg.apache.nifi.bootstrap.config.log.dir=/var/log/nifi
-Dorg.apache.nifi.bootstrap.config.pid.dir=/var/run/nifi
-Dorg.apache.nifi.bootstrap.config.file=/opt/nifi-1.13.2/conf/bootstrap.conf
org.apache.nifi.bootstrap.RunNiFi start
nifi 51872 332 14.0 74496404 37058284 ? Sl 10:47 487:17
/usr/lib/jvm/java/bin/java -classpath
/opt/nifi-1.13.2/./conf:/opt/nifi-1.13.2/./lib/javax.servlet-api-3.1.0.jar:/opt/nifi-1.13.2/./lib/jetty-schemas-3.1.jar:/opt/nifi-1.13.2/./lib/logback-classic-1.2.3.jar:/opt/nifi-1.13.2/./lib/logback-core-1.2.3.jar:/opt/nifi-1.13.2/./lib/jcl-over-slf4j-1.7.30.jar:/opt/nifi-1.13.2/./lib/jul-to-slf4j-1.7.30.jar:/opt/nifi-1.13.2/./lib/log4j-over-slf4j-1.7.30.jar:/opt/nifi-1.13.2/./lib/slf4j-api-1.7.30.jar:/opt/nifi-1.13.2/./lib/nifi-api-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-framework-api-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-server-api-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-runtime-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-nar-utils-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-properties-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-stateless-bootstrap-1.13.2.jar:/opt/nifi-1.13.2/./lib/nifi-stateless-api-1.13.2.jar
-XX:+PrintGCDetails -Dcom.sun.management.jmxremote.rmi.port=30008
-XX:+UseGCLogFileRotation -Xloggc:/var/log/nifi/gc.log
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=30008
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.local.only=true -Duser.timezone=Europe/Zurich
-Dorg.apache.jasper.compiler.disablejsr199=true -Xmx31g -Xms8g
-Dcurator-log-only-first-connection-issue-as-error-level=true
-Djavax.security.auth.useSubjectCredsOnly=true
-Djava.security.egd=file:/dev/urandom -Dzookeeper.admin.enableServer=false
-Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true -XX:ReservedCodeCacheSize=256m -XX:+UseG1GC
-Djava.protocol.handler.pkgs=sun.net.www.protocol -XX:NumberOfGCLogFiles=10
-XX:+UseCodeCacheFlushing -XX:CodeCacheMinimumFreeSpace=10m
-XX:GCLogFileSize=10m
-Dnifi.properties.file.path=/opt/nifi-1.13.2/./conf/nifi.properties
-Dnifi.bootstrap.listen.port=39853 -Dapp=NiFi
-Dorg.apache.nifi.bootstrap.config.log.dir=/var/log/nifi org.apache.nifi.NiFi
nifi 53618 0.3 0.0 31964 23484 ? Ssl 09:11 0:44
/opt/pushgateway/pushgateway --web.listen-address 0.0.0.0:9091
{code}
We have a systemd services for NiFi, so in theory it should never start more
than one NiFi:
{code:java}
[ldr@p-li-nifi-05 ~]$ cat /etc/systemd/system/nifi.service
[Unit]
Description=Apache NiFi
After=network.target[Service]
Type=forking
User=nifi
Group=nifi
RuntimeDirectory=nifi
RuntimeDirectoryMode=0775
ExecStart=/opt/nifi/bin/nifi.sh start
ExecStop=/opt/nifi/bin/nifi.sh stop
ExecReload=/opt/nifi/bin/nifi.sh restart
LimitNOFILE=1000000[Install]
WantedBy=multi-user.target
{code}
> Timezone wrong in UI for an 8 node cluster
> ------------------------------------------
>
> Key: NIFI-8423
> URL: https://issues.apache.org/jira/browse/NIFI-8423
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
> Affects Versions: 1.13.2
> Environment: 8 Node NiFi Cluster on CentOS 7
> OpenJDK 1.8.0_282
> Local timezone: Europe/Zurich (CEST or UTC+2h)
> Reporter: Josef Zahner
> Priority: Critical
> Labels: centos, cluster, openjdk, timezone
> Attachments: image-2021-04-13-15-14-02-162.png,
> image-2021-04-13-15-14-06-930.png, image-2021-04-13-15-14-56-690.png,
> manual_configured_timezone_gui_output.png, nifi-app_log.png
>
>
> We just upgraded to NiFi 1.13.2 and Java 1.8.0_282
> On our 8 node NiFi 1.13.2 cluster with timezone Europe/Zurich (CEST/CET), we
> have the issue that the UI does display the correct timezone (CEST, so UTC
> +2h), but in fact the time is displayed as UTC. NTP is enabled and working.
> The OS configuration/location is everywhere the same (doesn't matter if
> single or cluster NiFi). My tests below are all done at around 15:xx:xx local
> time (CEST).
> As you can see below, the timezone seems to be correct, but the time itself
> within NiFi is 2h behind (so in fact UTC) compared to Windows. In earlier
> NiFi/java versions it was enough to multiple times restart the cluster, but
> on the newest versions this doesn't help anymore. It shows most of the time
> CEST with the wrong time or directly UTC.
> !image-2021-04-13-15-14-06-930.png!
>
> The single NiFi instances or the 2 node clusters are always fine. The issue
> exists only on our 8 node cluster.
> NiFi Single Node Screenshot, which is fine (CEST, so UTC + 2h):
> !image-2021-04-13-15-14-02-162.png!
>
> If we set the -Duser.timezone to "Europe/Zurich" in bootstrap.conf, the UI
> shows no summer time, so only GMT+1 instead of GMT+2. As well not what we
> want.
> {code:java}
> java.arg.20=-Duser.timezone="Europe/Zurich"{code}
> !image-2021-04-13-15-14-56-690.png!
>
> What Matt below suggested has been verified, all servers (single nodes as
> well as clusters) are reporting the same time/timezone.
> [https://community.cloudera.com/t5/Support-Questions/NiFi-clock-is-off-by-one-hour-daylight-savings-problem/td-p/192942]
>
> So the question remains, where on a NiFi cluster comes the time from the UI
> and what could cause it that it is wrong? Sometimes I get UTC, sometimes I'm
> getting CEST but the time is anyhow UTC instead of CEST... I really need to
> have the correct time in the UI as I don't know what the impact could be on
> our dataflows.
>
> Any help would be really appreciated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
