Set up a NiFi Cluster

This topic explains how to set up the NiFi nodes that will also run the embedded ZooKeeper server. After setting up these nodes you can add additional nodes, but the additional nodes do not need to run ZooKeeper.

To set up ZooKeeper nodes for a NiFi cluster

  1. Install Apache NiFi.
  2. Open the file ./conf/nifi.properties and set or review the following properties:

    Property Description
    nifi.state.management.embedded.zookeeper.start=true Specifies whether this node runs an embedded ZooKeeper server. Set this property to true.
    nifi.state.management.embedded.zookeeper.properties Specifies the path to the embedded ZooKeeper configuration file. You can use the default value.
    nifi.web.https.host The fully qualified domain name of the NiFi host machine, for example nifi1.example.com.
    nifi.web.https.port The NiFi web interface HTTPS port. This is the port that serves the Apache NiFi web interface.
    nifi.sensitive.props.key The key to use when encrypting and decrypting sensitive properties in NiFi dataflows. You can choose any value, but it must be the same on all of the nodes in the cluster.

    NiFi TLS Properties

    nifi.security.keystore=./conf/nifi-server.p12
    nifi.security.keystore.certificate=
    nifi.security.keystore.privateKey=
    nifi.security.keystoreType=PKCS12
    nifi.security.keystorePasswd=changeit
    nifi.security.keyPasswd=
    nifi.security.truststore=
    nifi.security.truststore.certificate=./conf/cacert.crt
    nifi.security.truststoreType=PEM
    nifi.security.truststorePasswd=

    To enable SSL/TLS you must provide either:

    • a keystore containing a server certificate and private key
    • or alternatively, a server certificate and private key located in separate files.

    The NiFi instances in a cluster need to communicate with each other. Each server should have its own private key and certificate, and each server's certificate must be trusted by the other nodes in the cluster.

    When deploying NiFi in production, an administrator could use SSL certificates signed by a trusted root CA (such as DigiCert) or their organization's own certificate authority.

    If you are deploying a NiFi instance for testing purposes, you might use self-signed certificates. In this case you could create your own CA, but ensure that you add your custom CA certificate to the truststore on every node.

    There are many tools that you can use to create SSL certificates. For an overview of the steps and example commands for OpenSSL, see Example NiFi SSL Configuration.

    nifi.cluster.protocol.is.secure=true Set this property to true.
    nifi.cluster.is.node=true Specifies whether this node is part of a cluster. Set this property to true.
    nifi.cluster.node.address The fully qualified domain name of the NiFi node, for example nifi1.example.com.
    nifi.cluster.node.protocol.port=11000 Specify any free port above 1024.
    nifi.cluster.load.balance.port=12000 Specify any free port above 1024.
    nifi.zookeeper.connect.string

    A comma-separated list of host names and ports, for connecting to each of the embedded ZooKeeper servers.

    The host names should be the host names of the NiFi nodes on which you have enabled the embedded ZooKeeper server.

    The port numbers must match the ZooKeeper secure client port that you set in zookeeper.properties on the relevant node. For example if you use port 4881 as the secure client port on every ZooKeeper node:

    nifi1.example.com:4881,nifi2.example.com:4881,nifi3.example.com:4881
    nifi.zookeeper.client.secure=true Whether to enable TLS when communicating with ZooKeeper. Set this property to true.

    NiFi ZooKeeper Security properties:

    nifi.zookeeper.security.keystore=./conf/nifi-server.p12
    nifi.zookeeper.security.keystoreType=PKCS12
    nifi.zookeeper.security.keystorePasswd=changeit
    nifi.zookeeper.security.truststore=./conf/cacert.crt
    nifi.zookeeper.security.truststoreType=PEM
    nifi.zookeeper.security.truststorePasswd=

    SSL configuration for the NiFi node (as a client) communicating with the ZooKeeper server(s). For example, the nifi.zookeeper.security.truststore is used to verify the identity of the ZooKeeper server(s).

    You can use the same keystore and truststore that you used for the nifi.security.* properties above.

  3. Open the file ./conf/state-management.xml and set the Connect String property (in the ZooKeeper section) to the same value you set for nifi.zookeeper.connect.string in the nifi.properties file. For example:

    <property name="Connect String">nifi1.example.com:4881,nifi2.example.com:4881,nifi3.example.com:4881</property>
  4. Open the file ./conf/zookeeper.properties.

    1. Set the following parameters:

      serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
      secureClientPort=4881
      admin.enableServer=false

      TIP: In zookeeper.properties you can set parameters that configure the key store and trust store for the ZooKeeper server, but this is not necessary. NiFi automatically configures the embedded ZooKeeper server using properties you set in nifi.properties. You only need to set these properties in zookeeper.properties if you want to use a different configuration for ZooKeeper than for NiFi itself.

    2. Use the server.N parameter to list the nodes that will belong to the cluster. For example:

      server.1=nifi1.example.com:2881:3881
      server.2=nifi2.example.com:2881:3881
      server.3=nifi3.example.com:2881:3881

      TIP: When you set up the other nodes, ensure that you use the same configuration.

      The port numbers on each row are used by ZooKeeper. You can specify any free ports above 1024.

      TIP: Do not add the client port to the server parameter, as in the following example:

      server.1=nifi1.example.com:2881:3881;4881
      server.2=nifi2.example.com:2881:3881;4881
      server.3=nifi3.example.com:2881:3881;4881

      This example specifies an insecure client port, and you will see the following message in the nifi-app.log:

      WARN [main] o.a.n.c.s.server.ZooKeeperStateServer Invalid configuration was detected: A secure NiFi with an embedded ZooKeeper was configured for insecure connections. Insecure ports have been removed from embedded ZooKeeper configuration to deactivate insecure connections
  5. Create a text file ./state/zookeeper/myid that contains the server index. In the previous step you edited zookeeper.properties and specified that host1 has a server index of 1 (because it is listed as server.1), host2 has a server index of 2 (server.2), and host3 has a server index of 3 (server.3). The text file that you create should contain only the number and nothing else.

  6. Repeat the previous steps for the other nodes in your cluster.
  7. Start Apache NiFi on all of the nodes in the cluster.
  8. Open the NiFi user interface for any node in the cluster.

    • The user interface shows this icon, describing the number of nodes in the cluster:
    • To check the status of the nodes in the cluster, click followed by Cluster.