z2-Environment

h1. What is z2

[[Wiki|home]] :: [[How_to_install_z2|next]]

h2. What is the z2-environment?

The z2-environment is a Java server environment that allows to develop and operate Java server applications (e.g. Web applications) in a new and particularly easy and consistent way.

So what's the difference compared to Java servers like Tomcat?

The z2-environment is not yet another j2ee container/servlet engine (it does actually uses jetty as servlet engine though), but it is a new approach on how to develop and operate Java servers and get rid off a huge amount of infrastructure beside the main product and the pain people have to keep this infrastructure alive.

h3. The main point

The main point of the z2-environment (which is also the main difference compared to other Java servers) is that it is directly attached to the VCS (Version Control System). The VCS (currently subversion and git are supported) contains the Java sources, Web application resources, and configuration files organized in modules (the modules usually correspond 1:1 to projects in your IDE). Z2 knows this module structure (including dependencies between modules) and can get an overview by simply scanning a configured root folder in the VCS. In the same way z2 easily findw out what has been changed since the last scan by examining the commit logs. As a last step z2 _pulls_ the changes (all resources and configuration files) into its working directories. These three simple VCS access methods allow z2 to consistently update itself in contrast to typical Java servers where you have to make sure to consistently (re-) deploy your changes (_push_ approach).

h3. Why _pull_ is better than _push_

Because...

# Pushing a new version of a deployable to a Java server might require to update dependent deployables as well. The question is often: Were do you get the right versions from? Pulling solves this problem, because all it needs is the trigger to scan the VCS for *all* changes committed by you and your colleagues. The combination of manual trigger plus automatic VCS scan is called a _z2 synchronization_.

# Pushing deployables includes serveral preparation steps: One has to compile source files and package results into deployable units (typically .war,  .ear ZIP archives). Z2 does not need these intermediate packages, because it understands the very same project structure of the very same VCS the developer is working on. There is actually no difference between how the developer accesses the VCS using an IDE and how z2 accesses it. Both are using some java libraries like jgit or svnkit and both are working on the same resources.

# The push approach easily leads to a divergence between the projects the developer is working on in his IDE and the programs and resources running on the runtime environment (as there is no 1:1 mapping anymore).  As described above, z2 uses the very same project structure as the developer, so that there is a 1:1 mapping between the projects in the IDE and the modules considered by the runtime environment. 

h3. But isn't there something missing?

You might wonder where the Java sources are compiled into binaries. The VCS contains only sources (apart from pre-compiled 3rd party  jar archives) and there is no build/make/deploy, so how does the JVM get its byte code to run?

Z2 compiles the sources on the fly (and on demand). The approach is similar to how Java servers handle JSPs for more than a decade now. Compilation, in particular at runtime when dependency code is typically already available in memory, has become amazingly fast and will typically not be noted causing any delay. Z2 uses the proven Eclipse Java Compiler. As z2 does not need to package resources into .war files or .ear files, time spent on ZIPping and unZIPping can be saved.  Sources will only be compiled if required by changes.

Hence, in productive mode sources are compiled once and the system runs for a (hopefully) long while before patches/upgardes are applied.

Thus compiling on the server is not a problem but rather a solution. As an additional benefit,  sources are always compiled with the same JVM version the server is using.

h3. Looking a level deeper

As mentioned before z2 uses the same project structure as the developer. But this is only the first level. Inside the projects z2 is organized into _components_. 

Components can be Java modules as well as web modules, database connections, VCS connections and more. Components have a two level name starting with the module or project name followed by a component name. These two names a separated by a slash. The module "com.xyz.webshop" may have the following components:

* com.xyz.webshop/java 

(containing the business logic, the database access layer, the web access layer in terms of servlets etc)

* com.xyz.webshop/web

 (containing the web atrifacts like JSPs, HTML files, Stylesheets, images, JS files etc)

* com.xyz.webshop/database 

(containing the database connection settings)

The "java" and the "web" components contain multiple files inside folders and sub-folders while the "database" component is just a single .properties file.

Java components typically depend on other Java components which are declared in the java component's configuration file.

Components have a livecycle: They can be loaded and released. A component is loaded when it is first requested, possibly as early as part of the bootstrap process. A component is released during a z2-synchronization according to changes of its resources in the VCS or due to changes of dependency components. This way Java sources which have become invalid are unloaded, new versions are fetch from the VCS and compiled again. At a final step the released components can be loaded again (either on demand or as part of a partial bootstep process which is executed after a synchronization step).

Last but not least: the z2 component model is extensible. One can write it's own component types by developing a component factory.

h3. Cloning the sources

The z2_base project on gitorious.org contains several repositories. This makes it easier to setup different systems for different purposes and allow to easily transport (or in git-terms 'fetch' and 'push') sources between systems (here: gitorous projects). While one system serves for the development of the z2 foundation (which is the purpose of this z2_base system) another system might have a productive focus of running a webshop (for example call it z2_webshop). 

The webshop would go into its own repository z2_webshop. Furthermore z2_webshop would contain clones of the core.git and base.git repository from z2_base. 

From time to time one can update z2_webshop with changes from z2_base. This will affect only the repositories core.git and base.git while webshop.git remains untouched. Furthermore one can make emergency fixes in z2_webshop/base.git and _transport_ them back to z2_base/base.git later on.  Git makes it very easy to build up transport paths between _base systems_ and _application systems_.

Another setup could be to have development systems, test systems and productive systems. Usually the source are transported (pushed) from the development system to the test system where they are tested and qualified. Finally they are pushed into the productive system. However it is possible to do emergency fixes in the test system or even in the productive system. Of course nobody wants to apply fixes to productive systems, in the same way nobody wants to become sick. But sh*** happens. In case this is necessary the changes must be commited into the VCS of the productive system. Thus the huge benefit is that this change is automatically recorded in the commit logs. That means the z2-environment is completely auditable!

A transport of sources does not necessarily mean to reboot the whole z2-environment. Usually a z2-synchronization step is sufficient. Of course some components have to be released and thus reloaded - this is the true for all live systems. However the z2-environment minimizes the down-times by only stopping and restarting those components which have been changed (plus there dependent components).

h3. The source is the system

The motto of the z2-environment is: _"The source is the system!"_. Existing systems with a push-deploy approach tend to become a heterogeneous conglomerate after a while, containing a mixture of original versions, patches, hotfixes and manual adjustments on config files. It is hard (or even impossible) to tell who installed that patch last month or who changed the memory settings yesterday. Often it is even difficult to tell what patch versions are actually deployed. And once this question is answered the next one pops up: Where do I get the corresponding sources for debug and maintainance?

When the "source is the system", these questions are answered immediately. Changes to the z2-environment are always changes in the repositories (there is one major exception to this rule which is explained in [[First_steps_with_z2]]. And changes in the repositories are logged in the commit logs with name and date, regardless if it's a fetch from another system (which is the equivalent to a patch) or a manual fix on configuration files. Also exactly the same sources are available, because there is only one source for (Java) sources which is the repository. When the Java sources are changed, z2 has to recompile them in order to get them executed. Thus it is always possible to debug a system with the most up-to-date sources.

[[Wiki|home]] :: [[How_to_install_z2|next]]