Reduce Repo Traffic - Pre Deploy-Build and Offline Execution » History » Version 1
Henning Blohm, 30.12.2021 21:47
| 1 | 1 | Henning Blohm | h1. Reduce Repo Traffic - Pre Deploy-Build and Offline Execution |
|---|---|---|---|
| 2 | |||
| 3 | As a principle, a z2 installation is made to check for updates at startup and on-demand. After installation of the core however initial repository access may overwhelm the repository infrastructure leading to reduced scale out performance. In other cases repository access from a live installation is not desired. |
||
| 4 | |||
| 5 | h2. Example Problems |
||
| 6 | |||
| 7 | * You have large Git Repos and initial cloning by the system takes long |
||
| 8 | * You use the Maven Component Repository and do not want live-systems to download from a public maven repo |
||
| 9 | * You want to use z2 in a docker image for scale out in e.g. Kubernetes and want to avoid lengthy initialization times |
||
| 10 | * You want to always run live systems in offline mode, and rather than having z2 fetch updates, build a complete z2 installation for distribution |
||
| 11 | |||
| 12 | All of these cases are made up from a mix of the following concerns: |
||
| 13 | |||
| 14 | a) Initialization time and network usage of a fresh core instance |
||
| 15 | b) Avoiding access to remote systems at runtime |
||
| 16 | |||
| 17 | h2. Reduce Initialization Time and Network Usage of a Clean Z2-Home Installation |
||
| 18 | |||
| 19 | h3. Pre-Deploy Build |
||
| 20 | |||
| 21 | Assuming you want to quickly scale out and start up instances without requiring downloading of resources it is simplest to actually build a "warm" z2-core by retrieving all component resources and possibly even prebuilding all Java components. For example, this could be the last step of preparation of a docker image for distribution. |
||
| 22 | |||
| 23 | See below Appendix 1 for a code sample. |
||
| 24 | |||
| 25 | Pros: |
||
| 26 | * Simple to implement and fits nicely to a docker setup |
||
| 27 | |||
| 28 | Cons: |
||
| 29 | * Image could still be large, if repos hold a lot of unused content/history |
||
| 30 | |||
| 31 | h3. Hub CR |
||
| 32 | |||
| 33 | Another way of generally reducing and optimizing resource retrieval for a larger distributed system is to use the Hub Component Repository (see [[How_to_use_the_hub_cr]]). In essence, your system will have different target states depending on whether it is used on a development scenario or for production. In the latter case, the Hub CR would be used and otherwise regular source code repositories would be used - controlled by e.g. environment vars. This approach can be combined with the Pre-Deploy Build. |
||
| 34 | |||
| 35 | Pros: |
||
| 36 | * Only what is needed is retrieved - optionally pre-compiled |
||
| 37 | |||
| 38 | Cons: |
||
| 39 | * Non-Trivial repo or target state configuration depending on scenarios |
||
| 40 | * Requires a Hub setup and installation. Another central piece of infrastructure. |
||
| 41 | |||
| 42 | h3. SFTPSSH Repo |
||
| 43 | |||
| 44 | Finally, another approach would be to use the [[SFTPSSH_Component_Repository]] (that has not been implemented as of now). |
||
| 45 | |||
| 46 | Pros: |
||
| 47 | * Nodes only download what is needed - in pre-packaged form |
||
| 48 | * Simple to setup and maintain. No active infrastructure besides some SFTP location. |
||
| 49 | |||
| 50 | Cons: |
||
| 51 | * Non-Trivial repo or target state configuration depending on scenarios |
