z2-Environment

h1. How to use the Hub Component Repository (beta)

h2. Principles

(See also "Z2V2.1 is out – working on v2.2":http://www.z2-environment.net/blog/2012/12/z2v2-1-is-out-working-on-v2-2/)

By default, a Z2 installation (a z2 Home) accesses the version controlled storage that holds the system definition directly to download and prepare (and compile) modules as needed. This is the underpinnings of the system centric, pull-deployment based approach. In some situations however, it is not desirable to have source code on production servers at any time. This may be for compliance reasons or for fear of risking theft of intellectual property.

This is where the HubCR comes in. As a principle, all modules, all component resources, essentially anything the Z2 runtime knows about is served by component repository implementations.  There are implementations for Subversion, Git, file system folders, development workspaces and now, the latest addition, for another Z2 server that provides a consolidated, source-code free and pre-compiled view onto production resources (the HubCR provider).

So, instead of having production systems read and process source code directly, an intermediate node provides a semantically equivalent but pre-processed view onto the system definition:

p=. !{width:600px;}hubcr2.png!

The way the HubCR does that is by maintaining a pre-compiled and source-code stripped snapshot of the original production configuration. At the same time, the HubCR is just a regular z2 Home that runs the production config.

To the real production nodes (on the right in the diagram) however, the HubCR presents everything but the HubCR and other remote component repositories.

As a result, production systems can be completely separated from the source level details of the system definition. They don’t even see authentication details to the configuration store – only those necessary to access the HubCR service. At the same time, the pull semantics are preserved and updates can flow in and will be distributed consistently as for any other Z2-based system.

h2. How to configure and use

The HubCR consists of three parts: For once, there is a repository client implementation that is now part of the Z2 core (necessarily so that bootstrapping from a HubCR is possible). Secondly there is a z2-base level module that implements the serving side and thirdly there are some important rules that allow to implement the HubCR in a system that can be used in the default development flow as well.

The latter is a crucial cool feature: Just because a production system makes use of the HubCR, from where we do have access to the configuration source (i.e. the left side of the picture), we can still supply, implement and verify hot fixes without going through the HubCR.

h3. Consuming from a Hub

In order to connect a z2 <home> to a HubCR, you simply configure a repository to read from as in all the other cases. In case of the HubCR however, the home would only read from one repository (whereas the HubCR serving side may read from many). A HubCR consuming declaration of @com.zfabrik.boot.config/hubRepository.properties@  (see in particular [[How to create your own system]] to learn about repository topology) may look like this:

<pre><code class="bash">

com.zfabrik.systemStates.participation=com.zfabrik.boot.main/bootrepo_up

#

# hub stored component repository

#

com.zfabrik.component.type=com.zfabrik.hubcr

#

# The URL to the source repo. Typically this is of the form http://<server>:<port>/z_hubcr

#

hubcr.url=http://myhub:8080/z_hubcr

#

# Connection timeout in ms for connecting to the providing side (defaults to 10000)

#

hubcr.timeout=10000

#

# The remote user to use (optional but required by default)

#	

hubcr.user=hubcr_accessor

hubcr.password=admin

#

# The priority of the repository (defaults to 500)

#

hubcr.priority=500

</code></pre>

As mentioned above, when configuring a HubCR in a home, you will typically not connect to any other repository. So check carefully, if there are other effective repository configurations in @Z2_HOME/run/local/com.zfabrik.boot.config@.

See also the "ComponentRepositoryImpl":http://www.z2-environment.net/javadoc/com.zfabrik.boot.hubcr!2Fjava/impl/com/zfabrik/impl/hubcr/ComponentRepositoryImpl.html javadocs.

h3. Serving from a Hub

On the serving side, i.e. the node in the middle of the diagram above, the following configurations need to be applied:

# The Web application *com.zfabrik.hubcr/web* needs to be enabled, typically by a state dependency. This configuration needs to be applied in your system repository at (equivalently!) "com.zfabrik.hubcr/web/z.properties":https://redmine.z2-environment.net/projects/z2-base/repository/base/revisions/master/entry/com.zfabrik.hubcr/web/z.properties (or - locally - in a dev repository)

# Secondly the operational configuration at *com.zfabrik.hubcr/manager* should be revisited. As above, this needs to be done in your system repository at (equivalently!) "com.zfabrik.hubcr/manager.properties":https://redmine.z2-environment.net/projects/z2-base/repository/base/revisions/master/entry/com.zfabrik.hubcr/manager.properties (or - locally - in a dev repository). Details are given in the "Javadocs":http://www.z2-environment.net/javadoc/com.zfabrik.hubcr!2Fjava/impl/com/zfabrik/impl/hubcr/store/HubCRResource.html. 

One important concept to understand is how new revisions of the contents the HubCR serves come into existence. It is important to understand that not every update in the repositories that are read from the HubCR server create a new revision for HubCR consumers. As the HubCR needs to provide a consistent view onto its repository content, regardless of when a consumer asks for a specific resource, the HubCR builds a complete repository content structure at specific points in time to be served to its clients.

There are two alternatives:

h4. hubcr.scanOnSync=true

In that case, every synchronization of the HubCR serving Z2 updates the HubCR provided repository content. This procedure is simple and effective. 

h4. hubcr.scanOnSync=false

If you would prefer to sync the HubCR server without effecting consumers, you can choose this mode. In that case, updates on the HubCR repository content can be triggered from the (very basic) HubCR management interface (same login as /adm, user "z*" with password "z" by default) at "http://<server>:<port>/z_hubcr":http://localhost:8080/z_hubcr:

p=. !{width:600px;}manage_hubcr.png!

Finally, the JMX MBean @zfabrik:type=com.zfabrik.impl.hubcr.store.HubCRResource@ offers methods to check HubCR status and to trigger an update.

h3. One system, many homes with different provisionings

This final section is on the important rules (you should be prepared for that by now) that make sure, it all plays out.

If the HubCR would reflect the contents of the repositories readable by the HubCR server faithfully - modulo pre-compiling and the removal of source code - all repositories, such as add-on repositories that we like to configure in the environment module, would show up in the stock of modules and components visible to HubCR consumers (right side of the diagram). If that would happen, the HubCR consumer would try to read from them after all (which is exactly what we do not want - besides the point that the HubCR already provides everything those other repositories can offer).

In order to prevent that, the HubCR manager config (as mentioned above and "here":http://www.z2-environment.net/javadoc/com.zfabrik.hubcr!2Fjava/impl/com/zfabrik/impl/hubcr/store/HubCRResource.html) allows to specify name prefices and component types to filter out when preparing the HubCR content. The default is to filter all other repository types.

This way, consumers of the HubCR get a consistent view, without trying or requiring to access source-code repositories, while direct (you may say classical) consumers can still read from the source code repositories and see the system in its original glory as is the original promise of Z2.

h3. Configuration summary

To sum things up, in order to configure the use of the HubCR, you need to:

# On every HubCR consumer home replace the base and scenario repositories (@com.zfabrik.boot.config/baseRepository.properties@ and @com.zfabrik.boot.config/scenarioRepository.properties@ resp.) by a single HubCR repository (com.zfabrik.boot.config/hubcrRepository.properties) as described in [[#Consuming from a Hub]].

# Configure the HubCR Web app and manager in the module @com.zfabrik.hubcr@ in the your base repository (or - locally - in a dev repository) as described in [[#Serving from a Hub]].