How to configure and use the Gateway module¶

The Gateway module implements a "zero-downtime-upgrade" feature in Z2. Specifically, it uses the worker process management of Z2 in conjunction with an intermediate reverse proxy style Web handler to implement the following feature:

Upgrading a stateful Web application, i.e. a Web application that stores user data in its HTTP session typically implies downtime, and if the session state is not serializable and persisted during the upgrade, it does additionally imply that user state gets lost and typically that users need to log on again.

Using the Gateway, running sessions may be preserved and worker resources may still be assigned on the current software revision for as long as there are running sessions during a node upgrade and until all sessions have been terminated. The typical application of this feature is to roll out functional and user interface corrections without interrupting users. Users can switch over to post-upgrade software by terminating their session (e.g. via a log out) and starting a new one (e.g. by logging in again).

Note:

• This feature is relatively new and before you use it in production, you should have carefully tested your scenario.
• There are natural limitations to this feature. Upgrades that change the structure or semantics of persisted data or other resources that are shared across nodes cannot be handled this way.

A sample on how to use Gateway¶

As described in How to run a sample please clone the repository z2-samples.gateway and import the contained environment module. This module is holding a Gateway configuration as described below. After re-starting your Z2 installation, try the following:

1. Open a browser and navigate to http://localhost:8080/z_gateway. Use (by default) user "z*" with password "z".

You should see this:

2. Open another browser window and navigate to http://localhost:8080/adm (same user). Choose the group "Workers" and update.

Check for the current worker process and their state. You should see something like this:

Both web applications application create a HTTP session. Now go back to the Gateway user interface and press "Detach environment/webWorker@0 and sync". On your console you will see that another worker process started (called environment/webWorker@1) and if you update the worker list in the admin interface you should see something like this:

That is: One worker was detached while another one is in state started.

If you now refresh the Gateway user interface you will see that it is still served from environment/webWorker@0. Press "log off". You should find that it now switched to environment/webWorker@1.

Let's recap: Now we have a session on environment/webWorker@0 via the admin user interface and one one environment/webWorker@1 via the Gateway user interface.

3. Detach environment/webWorker@1 by clicking "Detach environment/webWorker@1 and sync" on the Gateway user interface.

When checking the worker list you will now see something like this:

Remember that it is the Gateway user interface that keeps environment/webWorker@1 alive. If you click on "log off" the session will be terminated, worker environment/webWorker@1 served its purpose and will terminate, and the Gateway user interface will be served from environment/webWorker@2. Checking the worker list you should see something like this:

Finally, if you wait 30 minutes until the session of the admin user interface has expired, also environment/webWorker@0 will terminate.

Enabling Gateway¶

The sample configuration in z2-samples.gateway has a pre-configured Gateway setup. Here is how this differs from a regular non-Gateway setup:

The Gateway Web server¶

As explained in the next section, when using Gateway, Z2 runs a Web server on the Home process in addition to Web servers in Web workers. All requests that go normally directly to the Web worker process will instead be handled on the Home process and forwarded appropriately to a Web worker process.

In order to enable this Web server, uncomment the system state participation in environment/gateway/z.properties as done in the sample.

The Gateway Web server is configured in environment/gateway/jetty.xml. This is a regular Jetty configuration. Instead of being an ordinary Java Web container all requests are handled by a special Gateway Jetty handler.

The only part of this configuration that could be of interest to be changed is the normal connector settings (ports and concurrency) - just as you normally would tune your Jetty Web server - and the name of the worker process that is to receive the forwarded requests - the Web worker process. The latter is defined in this section:

<Set name="handler">
  <New id="Handlers" class="org.eclipse.jetty.server.handler.HandlerCollection">
    <Set name="handlers">
      <Array type="org.eclipse.jetty.server.Handler">
        <Item>
          <New id="Contexts" class="org.eclipse.jetty.server.handler.ContextHandlerCollection"/>
        </Item>
        <Item> 
          <Call class="com.zfabrik.gateway.GatewayFactory" name="getHandler">
            <Set name="workerProcessComponentName">environment/webWorker</Set>
          </Call>
        </Item>
      </Array>
    </Set>
  </New>
</Set>

Unless you want to change the name of the target worker process, you do not need to touch Gateway's jetty.xml.

The Web worker Jetty configuration¶

When enabling Gateway, Web workers should not listen on the same port (default 8080) as Gateway. Also, they should process Gateway requests - which are slightly different than normal HTTP requests. In order to facilitate that, the normal Web server jetty.xml needs to be modified to use the GatewayConnector rather than any of Jetty's normal HTTP connectors. This is done in environment/webServer/jetty.xml by commenting the default setting and uncommenting the Gateway configuration. The relevant section looks like this:

<!-- Use the gateway connector, when using the Gateway feature. See also gateway/jetty.xml --> 
<Call name="addConnector">
  <Arg>
    <Call class="com.zfabrik.gateway.GatewayFactory" name="getConnector">
      <Set name="Acceptors">4</Set>
      <Set name="host">127.0.0.1</Set>
      <Set name="lowResourcesConnections">20000</Set>
      <Set name="lowResourcesMaxIdleTime">5000</Set>
    </Call>
  </Arg>
</Call>

Running the Gateway user interface¶

Finally, in order to actually trigger "Detach & Sync", you should enable the Gateway user interface by adding a dependency to com.zfabrik.gateway/web to environment/webWorkerUp.

How does Gateway work¶

Gateway consists of two parts:

The reverse proxy Gateway handler¶

When using Gateway, there is a Jetty Web server running on the Home process. This Web server maintains a mapping of sessions to Web worker nodes. Any newly created session will be assigned to the latest running Web worker process. All request will be routed to the Web worker process assigned to their session or, if there is no session id on the request, to the latest Web worker process.

On every Web worker process there is another Web server that accepts requests forwarded from the Home process and subsequently treats them as if they were ordinary HTTP requests in the first place.

Detached worker processes¶

Apart from being stopped, starting, started, or stopping, Z2 worker processes can also be in detached state. When a worker process is detached, it will terminate itself unless there is still unfinished work. Open Web application sessions is one type of such unfinished work.

In other words, if a worker has no open sessions and is being detached, it will terminate itself right away.

Finally, when the Home process runs a synchronization, detached worker processes do not count as running processes. That is, when the synchronization finds that there is only detached worker process instances of a worker process component, it will start another one.

That is why the Gateway user interface will not only detach a worker process but also trigger a synchronization. Here is a diagram illustrating the flow from detach and sync to worker process termination:

Ports of worker processes¶

Worker processes open ports to receive Web requests but also for JVM debugging and JMX access. These ports are configured in the worker process component definition. See for example environment/webWorker. These ports are actually base port numbers and the real port to be used is computed from a generation number that is also part of the worker process instance name that you saw above. I.e. the instance name environment/webWorker@2 is generation 2 of the worker process component environment/webWorker.

The port configurations in worker process components are:

Property	Description	Typical value
worker.debug.port	Base port of the JVM debug port to use. If the home process has remote debugging enabled, worker processes will also be configured for remote debugging.	5100 in environment/webWorker
worker.jmx.port	Base port of the remote JMX access. Will be ignored if no JMX access configured	7800 for environment/webWorker
com.zfabrik.gateway.port	Base port of the web worker side of Gateway. Will only listen to localhost and only be used, if Gateway is configured	8800 in environment/webWorker