Builder
An interface to a software builder.
- Overview
- Design
- Constraints
- Attributes
- Commands
- Related Types
Overview
Builder:
An interface to a software builder.
Intent
- Encourage abstracting software building so that a single abstraction can be used as an interface with a variety of build tools.
- Define an object that encapsulates the life-cycle of a software build.
- Separate the concerns of software building from software staging.
Problem
We want to design a build tool agnostic, software build life cycle process, but differences between software source code repositories, build tool setup and syntaxes demonstrate that inflexible and diverse procedures often ensue.
Discussion
The application build life-cycle is dominated by processing software resources using various tools. Primarily, the life cycle tasks fall into several phases:
- Preparing: Obtaining the necessary source materials
- Compiling: Compiling the source code into an executable form
- Testing: Invoking related unit tests
- Packaging: Packaging the build artifacts into a distributable format
- Staging: Storing the packaged build artifact into a repository for later deployment
Each phase of the build life cycle often entails a different set of underlying tools, procedures, order of tasks, configuration and application specific detail. The successful result of each of these phases is often a dependency for the subsequent phase in the cycle.
Ultimately, the goal of the build life cycle is to produce a set of release artifacts that will be later used in the deployment life cycle. Depending on the development project, these build artifacts might be code binaries, application presentation content, and reference data and database scripts.
The figure below describes these different dimensions to a build process:
Further complicating the requirements of the build infrastructure, is the need to produce multiple application components that integrate into a larger software system. These application components may occupy different architectural tiers and may even be developed by different teams. When one considers the build process for each of these application components, one can imagine the range of different technologies, inputs and outputs that each component requires. If one attempts to automate the build process for a related set of application components, the diversity of their tools, tool syntax and configuration typically become the complicating obstacle. This complexity often perpetuates siloed processes that are only integrated by the efforts of individuals that ensure all builds have completed, assembling the results by hand.
An alternate approach would be to introduce an abstraction that provides a common interface to executing the build life cycle for each component. This common interface should define the standard process structure of the build life cycle, with each phase defined in terms of its own phase-specific steps. These phase-specific steps can then be mapped to tasks that integrate with the tools and configuration used to generate each of the application components.
This offers several advantages: The general build life cycle is decoupled from the application component specific build procedures. The abstract interface facilitates the possibility to define integrated processes that automate many related application component builds. One can extend the generic build life cycle process structure by defining derived classes.
Structure
The builder abstraction breaks down into a common design pattern comprised of the following types:
- Builder: provides parameters and operations that govern the build process life-cycle. Optionally, the Builder object can make use of packaging logic by making use of an available Package type described next.
- Package: The Package base type provides commands for repository upload and registration. Sub types of Package (such as zip and war types ) provide concrete implementations for create and extract which can also be taken advantage of by Builders. Builders can interact with Package types to prepare build artifacts for later deployment.
SCM Bindings
The first phase of the Build life cycle must obtain and prepare the resources needed in the compilation phase. Typically, these resources are fetched from a source code repository, and depending on source code configuration management policy, these resources may need to be identified by a label (or branch/baseline id).
Given the range of SCM systems, each with their own tool specific usage and syntax for checking out files, committing changes, creating labels and or branches (cf. baselines), etc., the Build workflow commands that interact with the SCM repository need to use the appropriate SCM client interface providing to it the needed parameters.
The diagram below describes the proposed approach for managing various bindings to SCM clients. The scmBinding attribute is used to specify which SCM client binding to import. The Builder command.xml file uses that value to load the SCM binding file, declared as an antlib.
Each of the standard SCM operations should be implemented using a convention describing a well known set of macrodef (http://ant.apache.org/manual/CoreTasks/macrodef.html) declarations. Bindings for specific SCM clients are maintained in their own antlib (http://ant.apache.org/manual/CoreTypes/antlib.html) file. Each of these macrodefs are in turn exposed as standard Builder commands.
A listing of the standard set of macrodefs follows:
| Name | Description | Parameters |
|---|---|---|
| scmAdd | Adds a new file to the SCM | path |
| scmCheckout | Obtain a copy of the specified path from SCM for editing | basedir, connection, module, label, comment |
| scmCommit | Save changed path to SCM | path, comment |
| scmMkLabel | Create and apply label to path | path, label |
| scmMkTag | Create branch with path | path, label |
Example
The Builder type defines an object that encapsulates the build life cycle procedures for an application component. The build of a java web application provides a simple and familiar use case that we can use as a representative scenario.
The figure below illustrates these parts. A Builder named "headlines" is defined to produce two packages - headlines-2.0.1.war and appbase-2.01.jar. It takes as input build configuration data providing key parameters describing such details like: source code management connection information, and source code module lists. The scmCheckout command is parameterized to obtain the source code from the the SCM repository. The runBuildScript command is parameterized to invoke the application build tool, Ant in this example. Finally, the repoImport command is parameterized to find the packaged build artifacts produced by runBuildScript to upload and register the packages as deployable resources.
Check List
- Identify the source code checkout procedure
- Discern the parameters to the builds cript
- Override the scmCheckout and runBuildScript commands in a derived Builder type
- Optionally, delegate the package creation to an exisitng Package class.
Rules of Thumb
Design
- Super Type
- Deployment
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
| Notification | false |
| Template Directory | |
| Data View | Children, proximity: 1 |
| Logger Name | Builder |
Constraints
Allowed Child Dependencies
- BuilderBuildFile 1
- BuilderBuildTarget 1
- BuilderImportMax 1
- BuilderImportMin 1
- BuilderPackageExtension 1
- BuilderPackageFilebase 1
- BuilderPackageInstallroot 1
- BuilderPackageName 1
- BuilderPackageType 1
- BuilderPackageVersion 1
- BuilderScmBinding 1
- BuilderScmConnection 1
- BuilderScmLabel 1
- BuilderScmModule 1
1: These types have a Singleton constraint. Only one instance may be added as a resource.
Allowed Parent Dependencies
- Node
- Updater
Attributes
Exported Attributes
| Name | Property | Description |
|---|---|---|
| basedir | deployment-basedir | Directory where source code is checked out |
| targetdir | deployment-install-root | Directory where build artifacts are written |
Defaults for Imported Attributes
| Name | Default | Description |
|---|---|---|
| buildFile | Build file to invoke | |
| buildTarget | Target to invoke | |
| importMax | 1 | maximum number of packages to import after build |
| importMin | 1 | minimum number of packages to import after build |
| packageExtension | .* | name or pattern of file name extensions to import |
| packageFilebase | default | name or pattern of filenames to import (minus file extension) |
| packageInstallroot | Directory built package should be extracted | |
| packageName | default | String used to name package object |
| packageType | Package | Object type to use when registering uploaded packages |
| packageVersion | ${opts.buildstamp} | String to use for assigning version for package registration |
| scmBinding | cvs | binds scm commands to concrete SCM client commandsxs |
| scmConnection | connection string to access SCM repository | |
| scmLabel | name of the label/branch to access in SCM repository | |
| scmModule | name of code module in SCM repository |
Commands
Build
run the build cycle
The Builder life cycle is defined as a command called Build that is comprised of several phase specific steps; each of which are themselves encapsulated by standard operations: scmCheckout, runBuildScript and repoImport.
The figure below illustrates the generic process structure:
The -buildstamp argument is passed into each of the commands called by the workflow.
- Usage
- Build [-buildstamp <>]
Workflow
Options
| Option | Description |
|---|---|
| buildstamp | build identifier |
repoFind
find package objects in the repository
The repoFind command queries the project model maintained in the server for packages that match -packagetype, -version and/or -buildstamp. All results are stored in a named session.
If the -purge option is specified, the results will be passed as input to the repoPurge for removal from the repository.
- Usage
- repoFind [-buildstamp <.*>] [-packagetype <>] [-purge] [-session <find.session>] [-version <.*>]
Options
| Option | Description |
|---|---|
| buildstamp | build identifier (regex allowed) |
| packagetype | type (or subtype) of package to find |
| purge | run repoPurge using results of find |
| session | store results in session |
| version | version of packages to find (regex allowed) |
repoImport
import packages into the repository
The repoImport command recursively scans directories under -targetdir looking for files that match the regexes: (filebase).(extension) loading them into the repository if the necessary metadata is provided.
There are two setup modes for repoImport:
- Single package import: If only one build artifact is generated by the Builder, then ensure the follwoing: -min and -max both use a value of "1", -type refers to an existing Package type, -installroot refers to a valid file path.
- Multiple package import: If multiple build artifacts are generated by the Builder, then it is possible to use a -propfile as a metadata template for each kind of package to import.
The command will fail if the -min and -max criteria are not met.
- Usage
- repoImport [-buildstamp <>] [-extension <>] [-filebase <>] [-installroot <>] [-max <>] [-min <>] [-propfile <>] [-targetdir <>] [-type <>] [-version <>]
Options
| Option | Description |
|---|---|
| buildstamp | build identifier |
| extension | package file extension |
| filebase | package file base |
| installroot | package install-root |
| max | maximum number of packages to import after build |
| min | minimum number of packages to import after build |
| propfile | metadata template file used during package registration |
| targetdir | top level directory containing build artifacts |
| type |
package type
This must be a pre-existing type in the project model. |
| version | package version |
repoPurge
remove package objects from the repository
Processes results of repoFind and removes each package from the repository - deleting the file from the webdav and object in the project model.
The -packagename and -packagetype arguments allow one to filter the repoFind result data, only purging objects matching those patterns.
- Usage
- repoPurge [-packagename <.*>] [-packagetype <[^\.]*>] [-session <find.session>]
Options
| Option | Description |
|---|---|
| packagename | package name |
| packagetype | package type |
| session | session results are stored |
runBuildScript
Invokes the build tool
Invokes the build tool providing needed parameters. This is an empty implementation intended to be overriden by Builder subtypes.
- Usage
- runBuildScript [-basedir <>] [-buildfile <>] [-buildstamp <>] [-target <>] [-targetdir <>]
| Execution | bash |
| Arguments | echo implement run buildscript procedure |
Options
| Option | Description |
|---|---|
| basedir | directory where build resources reside |
| buildfile | build file to execute |
| buildstamp | build identifier |
| target | build target to evaluate |
| targetdir | directory build artifacts will be written |
scmCheckout
runs the checkout command
Checks out files into -basedir of -module via the -connection using method -binding.
- Usage
- scmCheckout [-basedir <>] [-binding <>] [-buildstamp <>] [-connection <>] [-label <>] [-module <>]
Options
| Option | Description |
|---|---|
| basedir | build directory |
| binding | scm binding |
| buildstamp | build identifier |
| connection | scmConnection identifier |
| label | source label to checkout |
| module | source module to checkout |
Related Types
The following types are defined for use with Builder.
BuilderSetting
Overview
BuilderSetting:
A builder setting.
Design
- Super Type
- Setting
| Role | Abstract. (Objects cannot be created.) |
| Instance Names | Unique |
Constraints
Allowed Parent Dependencies
BuilderScmConnection
Overview
BuilderScmConnection:
source code repository connection string
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| scmConnection | settingValue |
BuilderScmBinding
Overview
BuilderScmBinding:
binds scm commands to concrete SCM client commands
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| scmBinding | settingValue |
BuilderPackageFilebase
Overview
BuilderPackageFilebase:
name or pattern of filenames to import (minus file extension)
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| packageFilebase | settingValue |
BuilderPackageExtension
Overview
BuilderPackageExtension:
name or pattern of file name extensions to import
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| packageExtension | settingValue |
BuilderPackageType
Overview
BuilderPackageType:
Type of package
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| packageType | settingValue |
BuilderPackageName
Overview
BuilderPackageName:
Name of package
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| packageName | settingValue |
BuilderPackageInstallroot
Overview
BuilderPackageInstallroot:
install-root path of package
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| packageInstallroot | settingValue |
BuilderPackageVersion
Overview
BuilderPackageVersion:
package version
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| packageVersion | settingValue |
BuilderBuildFile
Overview
BuilderBuildFile:
Build file the Builder should invoke
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| buildFile | settingValue |
BuilderBuildTarget
Overview
BuilderBuildTarget:
Build target the Builder should invoke
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| buildTarget | settingValue |
BuilderScmModule
Overview
BuilderScmModule:
Source module to build
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| scmModule | settingValue |
BuilderScmLabel
Overview
BuilderScmLabel:
Source label to build
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| scmLabel | settingValue |
BuilderImportMin
Overview
BuilderImportMin:
Minimum number of packages to import
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| importMin | settingValue |
BuilderImportMax
Overview
BuilderImportMax:
Maximum number of packages to import
Design
- Super Type
- BuilderSetting
| Role | Concrete. (Objects can be created.) |
| Instance Names | Unique |
Attributes
Exported Attributes
| Name | Property |
|---|---|
| importMax | settingValue |