z2-Environment » org.z2env

h1. Intro

This project is for the development of [[z2-environment:wiki|z2-environment]] version 3.0.

Version 3.0 is a major refactoring release of z2 with new repository code lines and complete package renaming.

Check out the roadmap at "roadmap":https://redmine.z2-environment.net/versions/27

h1. Essential Differences between v2.3 and v3.

* All the typical declarations and package names are no longer *com.zfabrik.<something>* but rather *org.z2env.<something>*

* The <z2 home> layout is completely new and more flexible

* The essential component types that can be _invoked_ from the core is *system state* and *main program*.

* While Jetty is still contained to support Eclipsoid and z2Unit, we support a pre-installed Tomcat as application container (and with TomEE would even get a Java EE profile)

* The core is lighter and more dedicated. 

* There will be substantially less built-in application support like

** a local JTA implementation 

** No worker suppport without a corresponding add-on.

The main task of v3 is:

* Make the lower parts of z2 more accessible and easier to understand: While z2 previously started System Centric, we now close the gap from the ordinary main program.

* Increase the OSS flavor of z2

Obvious TODOs:

* Test TomEE

* Fix Eclipsoid templates

* Support MVNCR, also for distributions

h1. Draft Intro Documentation

h2. Getting Started

To understand how z2 works, you need to understand at most what are z2 modules and components and what are component repositories and how these concepts interplay. 

Check out [[wiki#Understanding-Z2|understanding-z2]] for that.

It is simplest to start by running a simple Java main program in z2 or a simple Web App. 

Follow this trail:

* [[wiki#Building-z2env-core|Building the core]]

* [[wiki#Running-a-Hello-World-main-program|Running a Hello World main program]]

* [[wiki#Running-a-Hello-World-Web-App|Running a Hello World Web App]]

* [[wiki#Developing-with-Eclipse-or-IntelliJ|Developing with Eclipse or IntelliJ]]

* [[wiki#Going-Multi-Module|Going Multi Module]]

h2. Building z2env-core

(For now, we need to build the core).

h3. Pre-requisites

* You need Git installed

* You need a JDK, at least version 7

* You need the ANT build tool

First clone http://git.z2-environment.net/z2env.core

<pre>

git clone http://git.z2-environment.net/z2env.core

</pre>

Change into *org.z2env.core.main* and run 

<pre>

ant -Doutput=<z2home> -Dsetup=<setup> 

</pre>

where <z2home> is where you want to install z2 (the *z2 home* that is) and <setup> is a choice of pre-configured content, actually component repositories.

If you omit *output* it will default to *gen/dist*. If you omit *setup*, no content will be pre-configured will be modified. 

Check the folder *setups* for available setups of the core build. These can be local (all repos will be cloned locally) or remote (repos will be remoted).

In general this build always updates only boot, z.jar, roots. It will not remove anything.

Example to install/update a core only into ../../z2env:

<pre>

ant -Doutput=../../z2env 

</pre>

To install/update a web-basic setup (with tomcat and eclipsoid support) with updates from remote repos:

<pre>

ant -Doutput=../../z2env -Dsetup=web-basic-remote

</pre>

Check out [[wiki#Understanding-the-ltHOMEgt|Understanding the <HOME>]] to learn more about the generated structure.

h2. Running a Hello World main program

For now, we will assume you have a z2 core installed in <HOME>. In its purest form, all the core can actually execute is simple Java main programs. 

Being in a modular environment, even if we only want to say „Hello World“, we need a module. Let's call it the greeter module. In order to have a main program run by z2, we need to also declare a  component for it, as that is what we tell z2 to run (not a class – mind you – but rather a component in a module). Assuming we use the local component name hello, than the absolut component name is 

<pre>

greeter/hello

</pre>

Finally, as this component actually has implementation code, we need to declare a Java component. By convention this is 

<pre>

greeter/java

</pre>

Now, after installing in <HOME> the place to simply drop a module is under <HOME>/modules. All in all, here is the complete structure to create:

| <HOME>/modules/greeter/hello.properties | 

<pre><code class="ruby">org.z2env.component.type=org.z2env.main

component.className=greetings.Main

</code></pre> |

| <HOME>/modules/greeter/java/z.properties | 

<pre><code class="ruby">org.z2env.component.type=org.z2env.java

</code></pre> |

| <HOME>/modules/greeter/java/src.impl/greetings/Main.java |

<pre><code class="java">

package greetings;

public class Main {

  public static void main(String[] args) {

    System.out.println("Hello!");

  }

}

</code></pre> |

That gives us the two components of module greeter. To run this on the command line, run

<pre>

java -DcomponentName=greeter/hello -jar z.jar

</pre>

Now, obviously this sample is not worth a modular execution environment. But even from here, you could check your module in with some Git repository that is referenced from the roots and all other users of the same repo would be able to run your component without installing anything.

h2. Running a Hello World Web App

This introduction explains how to connect a z2 core to a given Tomcat installation and how to develop a simple Web Application.

Furthermore it describes how to use the Dev Repository and the Eclipsoid plugin for the Eclipse or the IntelliJ development environment.

h3. Pre-Requisites

* We assume a development workspace in *workspace*.

* Have a local Tomcat (7 or higher) installation.*

h3. Steps we are going to take

# Create a module with a simple Web app

# Make it visible to Z2

# Run the Web App in a Tomcat Web Container

At first get yourself a z2env core with the *web-basic-local* or *web-basic-remote* setup. Make sure it is in your *workspace*. If you follow [[wiki#Building-z2env-core|Building the core]] this means running 

<pre>

ant -Doutput=workspace/z2env -Dsetup=web-basic-remote

</pre>

(*Note:* If in doubt, use an absolute path for *workspace*)

Using your IDE create a new project *hello* in *workspace* with the following structure:

| java/z.properties | 

<pre><code class="ruby">

org.z2env.component.type=org.z2env.java

java.privateReferences=\

	javax.servlet

</code></pre> |

| web/z.properties |

<pre><code class="ruby">

org.z2env.component.type=org.z2env.tomcat.webapp

webapp.path=/hello

</code></pre> |

| web/WebContent/index.jsp |

<pre><code class="html"><html><body>Hello! Did you know that 7*7=<%=7*7%>?</body></html></code></pre> |

| LOCAL | <empty file> |

Your *workspace* folder now contains two subfolders *z2env* and *hello*.

So far, that's it.  We will touch on the LOCAL file later. 

When we run Z2 knowing this Web Application, it will attempt to start a Tomcat Web Container that will serve the application's content. To do that, it needs to find one. 

In order to tell Z2 where to look, you can either set an environment variable *CATALINA_HOME* (you might have already – it's the Tomcat standard) or specify an environment variable *catalina.home*. 

To be on the safe side and assuming the location is */home/me/tomcat*, we open a terminal, change into *workspace/z2env* and run

<pre>

java -Dcatalina.home=/home/me/tomcat -jar z.jar hello/web

</pre>

Note, unlike for the main program, the command line will not return. Instead you can enter „q“ to quit or „s“ to synchronize. But first let's open a browser and check out 

<pre>

http://localhost:8080/hello

</pre>

*Note:* What you just created is a prototypical z2 module structure containing a Java component *java* (holding implementation) and a Web component *web*. 

Instead of creating the structure yourself, you could use the Eclipsoid plugin (introduced below) for that:

* In Eclipse: New / Others / new z2...., 

* In IntelliJ: New / Module / z2....

... but beware of the pre-generated configuration. Where it says *com.zfabrik* change to *org.z2env*.

*Note:* Now on to that LOCAL file. Unlike for the Hello World main program, this time it was to so-called Dev Repo that made our module available to z2. 

The dev repo is a key component to the development approach with z2. It checks for all modules that have a LOCAL file and that are stored some levels deep, relative to the parent folder of the z2 home (by default), which not so coincidentally is our workspace.

Try this: 

# Remove LOCAL

# Enter „s“ for synchronization. 

# Check the URL above (should say „not found“)

# Create a LOCAL file again

# Synchronize

# Check again.

Similarly, from now on, changes made to the hello module will be picked up, whenever you synchronize.

h2. Developing with Eclipse or IntelliJ

In this section we will add some code to the hello Web app and see how to develop using an integrated development environment like the Eclipse IDE or Jetbrains' IntelliJ IDEA.

h3. Pre-Requisite

* Have an Eclipse or IntelliJ installation

* Install the Eclipsoid plugin from http://z2-environment.net/eclipsoid/update/site.xml or by searching for "z2" in the IntelliJ Plugin Repository respectively

h3. Steps we are going to take

# Complete the development workspace setup

# Add a simple Controller

h4. In Eclipse

Use the workspace *workspace*. Create a Java project *hello* where we created the *hello* module. That is, we use the existing module as project content. You should now have an Eclipse development workspace that has one Java project named "hello". Right-click on the project and choose "Transform into z2 Project". A small "Z" icon should not decorate the project icon. 

Add a source folder *java/src.impl* to the project.

Try "Arm Z2 Projects" and "Disarm Z2 Projects" from the context menu. You should see a LOCAL file showing up and disappearing as well as the "Z" decoration showing a green halo and going back to normal. 

Go into Window/Preferences, pick "z2-Environment". Change the port to 8081, set the user to "z*" and the password to "z". Done.

h4. In IntelliJ

Add a source folder *java/src.impl* to the module.

Start a new Project with root folder *workspace*. Press the "Manage z2-Modules" button in the toolbar. Import the *hello* module into your IntelliJ project. Open the Settings dialog, pick "z2-Environment". Change the port to 8081, set the user to "z*" and the password to "z". Done.

h4. IDE Indepently

Start z2 as in the previous section but make sure the development support is enabled. Use this command line:

<pre>

java -Dcatalina.home=/home/me/tomcat -jar z.jar org.z2env.dev.shared/ready hello/web

</pre>

(possibly you want to turn this into a shell script)

*Note:* The _system state_ *org.z2env.dev.shared/ready* will pull up all necessary development support components. Make sure this is part of your to-be-run list.

Regardless whether you are using Eclipse or IntelliJ, press the "Resolve Classpath from Z2" button in the toolbar. This will make sure that all required dependencies for Z2 modules, as far as Java code is concerned, will be resolved for your workspace. 

Now add a class *SimpleController* and change the following files:

| java/src.impl/hello/SimpleController.java | 

<pre><code class="java">

package hello;

import javax.servlet.*;

import javax.servlet.http.*;

import java.io.IOException;

public class SimpleController extends HttpServlet {

    @Override

    protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {

        String input = req.getParameter("input");

        if (input!=null && input.trim().length()>0) {

            req.setAttribute("output","You said: "+input);

        }

        req.getRequestDispatcher("/").forward(req,resp);

    }

}

</code></pre> |

| java/z.properties | 

<pre><code class="ruby">

org.z2env.component.type = org.z2env.java

java.privateReferences=\

  javax.servlet,\

  javax.servlet.jsp.jstl

</code></pre> |

| web/WebContent/WEB-INF/web.xml | 

<pre><code class="xml">

<?xml version="1.0" encoding="UTF-8"?>

<web-app id="hello" version="2.5"

		xmlns="http://java.sun.com/xml/ns/javaee"

		xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

		xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">

    <servlet>

        <servlet-name>controller</servlet-name>

        <servlet-class>hello.SimpleController</servlet-class>

    </servlet>

    <servlet-mapping>

        <servlet-name>controller</servlet-name>

        <url-pattern>/do</url-pattern>

    </servlet-mapping>    

</web-app>

</code></pre> |

| web/WebContent/index.jsp |

<pre><code class="html">

<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core_1_1" %>

<html>

<body>

  <c:if test="${not empty output}">

    <p><c:out value="${output}"/></p>

  </c:if>

  <form method="POST" action="<c:url value='/do'/>">

    Input: <input type="text" name="input"><input type="submit" value="go!">

  </form>

</body>

</html>

</code></pre>|

Make sure the *hello* module is armed (right-click) by having a LOCAL file.

Now synchronize your running z2 by either:

* Pressing the "Sync with z2-Environment" button in the toolbar

* Pressing ALT+y in your IDE

* Entering "s" on the command line running z2.

Go to http://localhost:8080/hello and check it out.

h2. Going Multi-Module

In this section we will extend the previous work by a simple service module. 

h3. Pre-Requisites

# Have all the results from the previous section ready.

# Start IDE and z2 as above.

h3. Steps we are going to take

# add a new module with a service API

# reference the module API from the *hello* module and invoke the service.

In your workspace, use Eclipsoid to create a new z2 Java module *goodbye*. Open *java/z.properties* and change all *com.zfabrik.* prefixes to *org.z2env*. Essentially the file only needs the line

<pre><code class="ruby">

org.z2env.component.type=org.z2env.java

</code></pre>

In our API we will only expose an interface and rather leave the implementation hidden in the implementation parts of the module. We do that, as we prefer a strong separation of API and implementation. This is not at all required but a conscious design choice. 

| java/src.api/goodbye/GoodbyeService.java | 

<pre><code class="java">

package goodbye;

public interface GoodbyeService {

	String sayGoodbye(String in);

}

</code></pre> |

| java/src.impl/goodbye/GoodbyeServiceImpl.java | 

<pre><code class="java">

package goodbye;

public class GoodbyeServiceImpl implements GoodbyeService {

	@Override

	public String sayGoodbye(String in) {

		return "Goodbye "+in+"!";

	}

}

</code></pre> |

| java/service.properties | 

<pre><code class="ruby">

org.z2env.component.type=org.z2env.any

component.className=goodbye.GoodbyeServiceImpl

</code></pre> |

| java/z.properties | 

<pre><code class="ruby">

org.z2env.component.type=org.z2env.java

</code></pre> |

In effect we created the following:

* A module *goodbye* that has 

* A Java component with API and Implementation source code

* An _any_ component *goodbye/service* that manages the service implementation (as we will see below)

Back in the *hello* module, we want to use the _goodbye service_. We apply the following changes:

| java/z.properties | 

<pre><code class="ruby">

org.z2env.component.type=org.z2env.java

java.privateReferences=\

    javax.servlet,\

    javax.servlet.jsp.jstl,\

    goodbye

</code></pre> |

| java/src.impl/hello/SimpleController.java | 

<pre><code class="java">

package hello;

import goodbye.GoodbyeService;

import java.io.IOException;

import javax.servlet.ServletException;

import javax.servlet.http.HttpServlet;

import javax.servlet.http.HttpServletRequest;

import javax.servlet.http.HttpServletResponse;

import org.z2env.components.IComponentsLookup;

public class SimpleController extends HttpServlet {

    @Override

    protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {

    	GoodbyeService gb = IComponentsLookup.CL.lookup("goodbye/service", GoodbyeService.class);

        String input = req.getParameter("input");

        if (input!=null && input.trim().length()>0) {

            req.setAttribute(

        		"output",

        		gb.sayGoodbye(input)

        	);

        }

        req.getRequestDispatcher("/").forward(req,resp);

    }

}

</code></pre> |

After you have changed the Java references in *java/z.properties* you might want to press Alt-R or the "Resolve Classpath from Z2" button in the toolbar to have the IDE resolve the dependency for the *goodbye* module and so that

your IDE will not complain about using the *GoodbyeService* interface.

Now synchronize your running z2 by either:

* Pressing the "Sync with z2-Environment" button in the toolbar

* Pressing ALT+y in your IDE

* Entering "s" on the command line running z2.

Go to http://localhost:8080/hello and check it out.

h3. Discussion

What you just implemented is a programmatic component lookup. The first time <code>IComponentsLookup.CL.lookup("goodbye/service", GoodbyeService.class)</code> was invoked, z2 searched for a factory for *org.z2env.any* components. This specific factory simply created a singleton instance of the Goodbye Service and checked whether the requested interface is implemented. As that held, it returned the actual instance.

In later sections (for now check out [[z2-environment:How_to_Spring]]) we will explain how to that extends to the Spring Framework and Dependency Injection in general.

Feel free to apply changes to the Goodbye Service implementation class and have them made effective by synchronizing Z2.

h2. Going System-Centric

TBD

Getting more real by adding a remote repo as system repo.

Develop by checking out modules and using the Dev Repo.

h2. Understanding the <HOME>

We call an installation of a z2env core a *<home>*. The file structure of the *<home>* consists of few but important locations

h3. Layout

| z.jar | The actual core implementation. This is precompiled.|

| boot/ | A component repository adding to the core and provides the most basic capabilities such as compiling Java code and running a main program. |

| roots/ | A file system component repository that is always registered by the core. This is typically filled with other repository components linking to remote or local component repositories. |

| modules/ | A file system repository declared under roots/ that is useful to locally augment a core installation.|

We use the modules repository for example for command line experiments and simple hacks. It is not as useful for regular development operations as is the dev repo.

h3. Folders and Properties

When running z2 from the command line, some logic is applied to derive where stuff is and some defaults apply:

|_. What |_. System property |_. Defaults |

| Z2 home - the installation location | org.z2env.home | If no system property is set, the environment variable Z2_HOME will be checked. If that is not set, the z2 home location will be derived from the location of the z.jar file |

| Dev Repo workspaces - where the Dev Repo checks for local modules | org.z2env.dev.local.workspaces | Defaults to ${z2.home}/.. assuming modules will be next to the home or one level deeper |

h2. Understanding Z2

Most of the concepts that make up z2 can be derived from the desire to efficiently develop standard Java Web Applications that are defined in a central system definition. Add to that the wish to avoid all that build complexity and the desire for a practically usable modularization approach.

Once you loose the ability to tweak some build script or configuration to describe how to package a Web app or some libraries, you need some structural hint and simple declarations providing sufficient information.

Secondy, assuming you want to apply changes at runtime, you need some way of determining what to unload at runtime, given some changed file. Again: Structural information needs to present that maps to runtime state. 

Finally, as we want some support for modular application development anyway, why not make modules our essential structure. As we have not only code but also Web Applications, Data Sources,  Background Jobs and much more to consider, we add components with type into the mix. Voila.

*Component Repositories* provides modules and components to z2. A component is prefixed by the module name:

<pre>

<global component name> = <module name>/<local component name>

</pre>

In fact, the typical component repository has a file system style structure that is made up exactly like that. 

Structure from repo root:

| /<module>/<cmp>.properties | A component <cmp> in module <module> that has no resources on its own.|

| /<module>/<cmp>/z.properties | A component <cmp> in module <module> that may have additional resources on its own.|

Note that some components, like Web applications and typically Java components have file resources, such as HTML files or Java source code files, while other components, such as main program declarations or data source declarations, so not.

There are some conventions around this naming scheme. In particular, by convention, whenever a component requires Java implementation, z2 will first look for a component named *java* of the same module. As for example in the Hello World example.

To give you an idea what this model translates to at runtime consider this: At runtime, the static component is turned into a Resource instance by a component type implementation (that, in fact, is provided by another, built-in component type). 

!uml1.png!

Dependencies, for example those between Java components, translate into dependencies between resources that are observed during invalidation of resources. This introduces invalidation and synchronization.

h3. A crucial concept of z2 is that of a synchronization. 

For fast development roundtrips and in some cases for general configuration updates, it is desirable to not require a complete restart of a Java Virtual Machine process but rather just to change runtime state accordingly. 

In z2 a synchronization describes the following process:

# Check for changes in component repositories

# Based on changes found, identify components affected and invalidate associated runtime resources incl. dependent resources.

# Try to attain target states

That is, whenever you do a change and trigger a synchronization from your IDE, this is exactly what happens: Identify what needs to be thrown away, make sure to remove dependent stuff as well,  bring up what needs to be running.

h3. A second crucial concept of z2 is that of *System Centric Development*

When working on a software system with many access points and high re-use on a service level, as is typical for business systems, staying quick integration with a team's changes is key. Component repositories can be fed from remotely accessible version control systems such as Subversion and Git. When connecting to a central, team-shared repository, every synchronization will make sure that you are up-to-date with the real source of truth. 

Using the dev repo, you can still isolate yourself from changes on part of the system that you are actively working on. After your modifications are completed, rebased, and locally tested, getting a clean and consistent state is just one synchronization away.

h3. The third and final concept is that of a compile-free environment

It is within the nature of the synchronization concept, that anything required to happen to have your software available for execution must have happened after synchronization completed. Most preparation steps are executed at first use. This includes the compilation of Java code. Z2 uses the Eclipse Java Compiler for that purpose.

h2. The Component Model

See [[z2-environment:Z2_Modules_intro]]

h2. Java Components

See [[z2-environment:Z2_Java_components_intro]]