Chris I am trying what you have suggested, while passing the init container params in values.yaml getting below error, can you please help to get around this issue. *Error: cannot load values.yaml: error converting YAML to JSON: yaml: line 202: did not find expected ',' or '}'* I am adding below init container config: tried to edit it in multiple ways no luck :(
initContainers: { name: nifi-init image: "apache/nifi:1.11.1" imagePullPolicy: "IfNotPresent" command: ['sh', '-c', '/opt/nifi/nifi-toolkit-current/bin/tls-toolkit.sh client -c nifi-ca-cs -t Mytesttoken12345 --dn "CN=$(hostname -f), OU=NIFI"','>','/opt/certs'] volumeMounts: - mountPath: /opt/certs/ name: certs } Created CA service as below: apiVersion: apps/v1 kind: ReplicaSet metadata: name: nifi-ca namespace: nifi labels: app: nifi-ca spec: # modify replicas according to your case replicas: 1 selector: matchLabels: app: nifi-ca template: metadata: namespace: nifi labels: app: nifi-ca spec: containers: - name: nifi-ca image: apache/nifi:1.9.2 ports: - containerPort: 8443 name: ca-client-port command: - bash - -c - | ../nifi-toolkit-current/bin/tls-toolkit.sh server -c nifi-ca-cs -t <token> --- # Create service for the nifi-ca replica set apiVersion: v1 kind: Service metadata: name: nifi-ca-cs namespace: nifi labels: app: nifi-ca spec: ports: - port: 8443 name: ca-client-port targetPort: 8443 selector: app: nifi-ca On Fri, Jul 24, 2020 at 10:13 AM Atul Wankhade <atul.wankhad...@gmail.com> wrote: > Hi Andy, > > Sorry for the confusion, Nifi is running inside a container on the > node(Image has java prebuilt). It seems I need to tweak the image to > generate the certs inside the container. I have done the same setup(worked > fine) On Azure where I used to generate the certs on VM itself for Node > Identity so I was trying the same on Kubernetes Node but no Java here. I am > new to K8S/Docker so limited by imagination I assume. TLS toolkit is part > of the NiFi image but nowhere documented as how to use it inside the > container(k8s env). > Need to explore more on what Chris said. > > Thank you guys > Atul > > On Thu, Jul 23, 2020 at 9:27 PM Andy LoPresto <alopre...@apache.org> > wrote: > >> Chris has a lot of good suggestions there. NiFi can accept certificates >> from any provider as long as they meet certain requirements (EKU, SAN, no >> wildcard, etc.). The toolkit was designed to make the process easier for >> people who could not obtain their certificates elsewhere. >> >> Maybe I am misunderstanding your statement, but I am curious why the >> toolkit can’t run on the node — if you don’t have Java available, how does >> NiFi itself run? >> >> Andy LoPresto >> alopre...@apache.org >> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >> He/Him >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> On Jul 23, 2020, at 12:35 AM, Chris Sampson <chris.samp...@naimuri.com> >> wrote: >> >> My suggestion would be to run the apache/nifi-toolkit image as another >> Pod within your k8s namespace and have it running as a TLS Server[1]. >> You'll probably need to do that separately from your Helm chart (I'm not >> familiar with Helm or this chart). >> >> Then connect to that from your NiFi instances as they start up, e.g. with >> an init-container based on the same apache/nifi-toolkit image using the TLS >> client function [1] to obtain the required TLS certificate files from the >> TLS Server. You can use an emptyDir [2] volume to pass the files from the >> init-container to the NiFi container within the Pod. >> >> If you run the TLS Server as a StatefulSet (or a Deployment) with a >> Persistent Volume Claim that backed by an external volume within your cloud >> provider (whatever the GKE equivalent is of AWS's EBS volumes), then the >> TLS Server can be setup with its own Certificate Authority that persists >> between Pod restarts and thus your NiFi certificates shouldn't become >> invalid over time (if the TLS Server is restarted and generates a new CA, >> then subsequent NiFi restarts would mean your NiFi cluster instances would >> no longer be able to communicate with one another as they wouldn't trust >> one another's certificates). >> >> >> An alternative, if it's available in your k8s cluster, is to use >> something like cert-manager [3] to provision certificates for your >> instances, then use an init-container within the NiFi Pods to convert the >> PEM files to Java Keystore or PKCS12 format as required by NiFi. >> >> >> [1]: >> https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#client-server >> [2]: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir >> [3]: https://github.com/jetstack/cert-manager >> >> >> *Chris Sampson* >> IT Consultant >> chris.samp...@naimuri.com >> >> >> >> On Thu, 23 Jul 2020 at 07:09, Atul Wankhade <atul.wankhad...@gmail.com> >> wrote: >> >>> Thanks a lot Andy for your reply, it definitely helped pinpointing what >>> is going wrong. I tried simulating the same with the docker image from >>> Apache and generating the keystore/truststore files on the Docker host. For >>> one node NiFi it worked fine. The problem comes when I am trying the same >>> on Kubernetes. Nodes in GKE have Container optimized OS (no pkg installer) >>> , so it does not support using NiFi tls-toolkit as Java cannot be >>> installed. Can you please give some pointers/workaround on how to solve >>> this issue with k8s? >>> Once the files are generated we can mount it using Host mount in the pod. >>> >>> Thanks again for your help :) >>> Atul >>> >>> On Tue, Jul 21, 2020 at 10:37 PM Andy LoPresto <alopre...@apache.org> >>> wrote: >>> >>>> Atul, >>>> >>>> I am not a Kubernetes/ingress expert, but that error is indicating that >>>> you specified NiFi should be secure (i.e. use TLS/HTTPS) and yet there is >>>> no keystore or truststore provided to the application, so it fails to >>>> start. NiFi differs from some other applications in that you cannot >>>> configure authentication and authorization without explicitly enabling and >>>> configuring TLS for NiFi itself, not just delegating that data in transit >>>> encryption to an external system (like a load balancer, proxy, or service >>>> mesh). >>>> >>>> I suggest you read the NiFi walkthrough for “Securing NiFi with TLS” >>>> [1] which will provide some context around what the various requirements >>>> are, and the Admin Guide [2] sections on authentication and authorization >>>> for more background. >>>> >>>> [1] >>>> https://nifi.apache.org/docs/nifi-docs/html/walkthroughs.html#securing-nifi-with-tls >>>> [2] >>>> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#security_configuration >>>> >>>> >>>> Andy LoPresto >>>> alopre...@apache.org >>>> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >>>> He/Him >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>> >>>> On Jul 20, 2020, at 11:58 PM, Atul Wankhade <atul.wankhad...@gmail.com> >>>> wrote: >>>> >>>> Hi All, >>>> I am trying to install NiFi with SSL on Kubernetes using >>>> Helm(cetic/nifi), Below is my values.yaml. I keep getting an error on NiFi >>>> containers as - Am I missing something? >>>> *Caused by: org.springframework.beans.factory.BeanCreationException: >>>> Error creating bean with name 'clusterCoordinationProtocolSender' defined >>>> in class path resource [nifi-cluster-protocol-context.xml]: Cannot resolve >>>> reference to bean 'protocolSocketConfiguration' while setting constructor >>>> argument; nested exception is >>>> org.springframework.beans.factory.BeanCreationException: Error creating >>>> bean with name 'protocolSocketConfiguration': FactoryBean threw exception >>>> on object creation; nested exception is java.io.FileNotFoundException: (No >>>> such file or directory)* >>>> >>>> VALUES.YAML: >>>> --- >>>> # Number of nifi nodes >>>> replicaCount: 1 >>>> >>>> ## Set default image, imageTag, and imagePullPolicy. >>>> ## ref: https://hub.docker.com/r/apache/nifi/ >>>> ## >>>> image: >>>> repository: apache/nifi >>>> tag: "1.11.4" >>>> pullPolicy: IfNotPresent >>>> >>>> ## Optionally specify an imagePullSecret. >>>> ## Secret must be manually created in the namespace. >>>> ## ref: >>>> https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ >>>> ## >>>> # pullSecret: myRegistrKeySecretName >>>> >>>> securityContext: >>>> runAsUser: 1000 >>>> fsGroup: 1000 >>>> >>>> sts: >>>> # Parallel podManagementPolicy for faster bootstrap and teardown. >>>> Default is OrderedReady. >>>> podManagementPolicy: Parallel >>>> AntiAffinity: soft >>>> hostPort: null >>>> >>>> ## Useful if using any custom secrets >>>> ## Pass in some secrets to use (if required) >>>> # secrets: >>>> # - name: myNifiSecret >>>> # keys: >>>> # - key1 >>>> # - key2 >>>> # mountPath: /opt/nifi/secret >>>> >>>> ## Useful if using any custom configmaps >>>> ## Pass in some configmaps to use (if required) >>>> # configmaps: >>>> # - name: myNifiConf >>>> # keys: >>>> # - myconf.conf >>>> # mountPath: /opt/nifi/custom-config >>>> >>>> >>>> properties: >>>> # use externalSecure for when inbound SSL is provided by >>>> nginx-ingress or other external mechanism >>>> externalSecure: true >>>> isNode: true >>>> httpPort: null >>>> httpsPort: 8443 >>>> clusterPort: 6007 >>>> clusterSecure: true >>>> needClientAuth: true >>>> provenanceStorage: "8 GB" >>>> siteToSite: >>>> secure: true >>>> port: 10000 >>>> authorizer: managed-authorizer >>>> # use properties.safetyValve to pass explicit 'key: value' pairs that >>>> overwrite other configuration >>>> safetyValve: >>>> #nifi.variable.registry.properties: >>>> "${NIFI_HOME}/example1.properties, ${NIFI_HOME}/example2.properties" >>>> nifi.web.http.network.interface.default: eth0 >>>> # listen to loopback interface so "kubectl port-forward ..." works >>>> nifi.web.http.network.interface.lo: lo >>>> >>>> ## Include additional libraries in the Nifi containers by using the >>>> postStart handler >>>> ## ref: >>>> https://kubernetes.io/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/ >>>> # postStart: /opt/nifi/psql; wget -P /opt/nifi/psql >>>> https://jdbc.postgresql.org/download/postgresql-42.2.6.jar >>>> >>>> # Nifi User Authentication >>>> auth: >>>> ldap: >>>> enabled: false >>>> host: ldap://<hostname>:<port> >>>> searchBase: CN=Users,DC=example,DC=com >>>> searchFilter: CN=john >>>> >>>> ## Expose the nifi service to be accessed from outside the cluster >>>> (LoadBalancer service). >>>> ## or access it from within the cluster (ClusterIP service). Set the >>>> service type and the port to serve it. >>>> ## ref: http://kubernetes.io/docs/user-guide/services/ >>>> ## >>>> >>>> # headless service >>>> headless: >>>> type: ClusterIP >>>> annotations: >>>> service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" >>>> >>>> # ui service >>>> service: >>>> type: LoadBalancer >>>> httpPort: 80 >>>> httpsPort: 443 >>>> annotations: {} >>>> # loadBalancerIP: >>>> ## Load Balancer sources >>>> ## >>>> https://kubernetes.io/docs/tasks/access-application-cluster/configure-cloud-provider-firewall/#restrict-access-for-loadbalancer-service >>>> ## >>>> # loadBalancerSourceRanges: >>>> # - 10.10.10.0/24 >>>> >>>> # Enables additional port/ports to nifi service for internal >>>> processors >>>> processors: >>>> enabled: false >>>> ports: >>>> - name: processor01 >>>> port: 7001 >>>> targetPort: 7001 >>>> #nodePort: 30701 >>>> - name: processor02 >>>> port: 7002 >>>> targetPort: 7002 >>>> #nodePort: 30702 >>>> >>>> ## Configure Ingress based on the documentation here: >>>> https://kubernetes.io/docs/concepts/services-networking/ingress/ >>>> ## >>>> ingress: >>>> enabled: false >>>> annotations: {} >>>> tls: [] >>>> hosts: [] >>>> path: / >>>> rule: [] >>>> # If you want to change the default path, see this issue >>>> https://github.com/cetic/helm-nifi/issues/22 >>>> >>>> # Amount of memory to give the NiFi java heap >>>> jvmMemory: 2g >>>> >>>> # Separate image for tailing each log separately >>>> sidecar: >>>> image: ez123/alpine-tini >>>> >>>> # Busybox image >>>> busybox: >>>> image: busybox >>>> >>>> ## Enable persistence using Persistent Volume Claims >>>> ## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/ >>>> ## >>>> persistence: >>>> enabled: false >>>> >>>> # When creating persistent storage, the NiFi helm chart can either >>>> reference an already-defined >>>> # storage class by name, such as "standard" or can define a custom >>>> storage class by specifying >>>> # customStorageClass: true and providing the "storageClass", >>>> "storageProvisioner" and "storageType". >>>> # For example, to use SSD storage on Google Compute Engine see >>>> values-gcp.yaml >>>> # >>>> # To use a storage class that already exists on the Kubernetes >>>> cluster, we can simply reference it by name. >>>> # For example: >>>> # storageClass: standard >>>> # >>>> # The default storage class is used if this variable is not set. >>>> >>>> accessModes: [ReadWriteOnce] >>>> ## Storage Capacities for persistent volumes >>>> # Storage capacity for the 'data' directory, which is used to hold >>>> things such as the flow.xml.gz, configuration, state, etc. >>>> dataStorage: >>>> size: 1Gi >>>> # Storage capacity for the FlowFile repository >>>> flowfileRepoStorage: >>>> size: 10Gi >>>> # Storage capacity for the Content repository >>>> contentRepoStorage: >>>> size: 10Gi >>>> # Storage capacity for the Provenance repository. When changing this, >>>> one should also change the properties.provenanceStorage value above, also. >>>> provenanceRepoStorage: >>>> size: 10Gi >>>> # Storage capacity for nifi logs >>>> logStorage: >>>> size: 5Gi >>>> >>>> ## Configure resource requests and limits >>>> ## ref: http://kubernetes.io/docs/user-guide/compute-resources/ >>>> ## >>>> resources: {} >>>> # We usually recommend not to specify default resources and to leave >>>> this as a conscious >>>> # choice for the user. This also increases chances charts run on >>>> environments with little >>>> # resources, such as Minikube. If you do want to specify resources, >>>> uncomment the following >>>> # lines, adjust them as necessary, and remove the curly braces after >>>> 'resources:'. >>>> # limits: >>>> # cpu: 100m >>>> # memory: 128Mi >>>> # requests: >>>> # cpu: 100m >>>> # memory: 128Mi >>>> >>>> logresources: >>>> requests: >>>> cpu: 10m >>>> memory: 10Mi >>>> limits: >>>> cpu: 50m >>>> memory: 50Mi >>>> >>>> nodeSelector: {} >>>> >>>> tolerations: [] >>>> >>>> initContainers: {} >>>> # foo-init: # <- will be used as container name >>>> # image: "busybox:1.30.1" >>>> # imagePullPolicy: "IfNotPresent" >>>> # command: ['sh', '-c', 'echo this is an initContainer'] >>>> # volumeMounts: >>>> # - mountPath: /tmp/foo >>>> # name: foo >>>> >>>> extraVolumeMounts: [] >>>> >>>> extraVolumes: [] >>>> >>>> ## Extra containers >>>> extraContainers: [] >>>> >>>> terminationGracePeriodSeconds: 30 >>>> >>>> ## Extra environment variables that will be pass onto deployment pods >>>> env: [] >>>> >>>> # >>>> ------------------------------------------------------------------------------ >>>> # Zookeeper: >>>> # >>>> ------------------------------------------------------------------------------ >>>> zookeeper: >>>> ## If true, install the Zookeeper chart >>>> ## ref: >>>> https://github.com/kubernetes/charts/tree/master/incubator/zookeeper >>>> enabled: true >>>> ## If the Zookeeper Chart is disabled a URL and port are required to >>>> connect >>>> url: "" >>>> port: 2181 >>>> >>>> *Complete stacktrace:* >>>> Caused by: org.springframework.beans.factory.BeanCreationException: >>>> Error creating bean with name 'clusterCoordinationProtocolSender' defined >>>> in class path resource [nifi-cluster-protocol-context.xml]: Cannot resolve >>>> reference to bean 'protocolSocketConfiguration' while setting constructor >>>> argument; nested exception is >>>> org.springframework.beans.factory.BeanCreationException: Error creating >>>> bean with name 'protocolSocketConfiguration': FactoryBean threw exception >>>> on object creation; nested exception is java.io.FileNotFoundException: (No >>>> such file or directory) >>>> at >>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359) >>>> at >>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108) >>>> at >>>> org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:648) >>>> at >>>> org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:145) >>>> at >>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198) >>>> at >>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100) >>>> at >>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511) >>>> at >>>> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481) >>>> at >>>> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312) >>>> at >>>> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) >>>> at >>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308) >>>> at >>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) >>>> at >>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351) >>>> ... 75 common frames omitted >>>> Caused by: org.springframework.beans.factory.BeanCreationException: >>>> Error creating bean with name 'protocolSocketConfiguration': FactoryBean >>>> threw exception on object creation; nested exception is >>>> java.io.FileNotFoundException: (No such file or directory) >>>> at >>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185) >>>> at >>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) >>>> at >>>> org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640) >>>> at >>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323) >>>> at >>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) >>>> at >>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351) >>>> ... 87 common frames omitted >>>> Caused by: java.io.FileNotFoundException: (No such file or directory) >>>> at java.io.FileInputStream.open0(Native Method) >>>> at java.io.FileInputStream.open(FileInputStream.java:195) >>>> at java.io.FileInputStream.<init>(FileInputStream.java:138) >>>> at java.io.FileInputStream.<init>(FileInputStream.java:93) >>>> at >>>> org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:66) >>>> at >>>> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:45) >>>> at >>>> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:30) >>>> at >>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178) >>>> ... 92 common frames omitted >>>> 2020-07-17 11:04:25,204 INFO [Thread-1] org.apache.nifi.NiFi Initiating >>>> shutdown of Jetty web server... >>>> 2020-07-17 11:04:25,214 INFO [Thread-1] >>>> o.eclipse.jetty.server.AbstractConnector Stopped >>>> ServerConnector@700f518a{SSL,[ssl, >>>> http/1.1]}{0.0.0.0:8443} >>>> 2020-07-17 11:04:25,214 INFO [Thread-1] >>>> org.eclipse.jetty.server.session node0 Stopped scavenging >>>> >>>> Any help to resolve this is appreciated. >>>> Atul Wankhade >>>> >>>> >>>> >>