ProjectBuilder

ProjectBuilder: ProjectBuilder is a utility type to assist developing and loading of source-based projects in the ControlTier system.

ProjectBuilder is a Builder used to build types and type libraries, load object data, even generate a Forrest documentation site. ProjectBuilder along with a set of properly prepared source files, is an alternative to the primarily graphical modeling tools offered by the ControlTier server's Workbench web application.

ProjectBuilder supports a typical edit, build, test methodology familiar to most developers wherein, a set of source files maintained in a source code repository are developed using familiar text editing tools, and ProjectBuilder builds and deploys them, so that the results can be tested.

The Workbench application can still act as a useful tool in this methodology. Since it manages the active model for the project, it allows a user to get an integrated view of the type object model. Besides offering a visulaization layer to the active model, Workbench can also assist in the code development process. A developer can let Workbench do some code generation and then copy those generated files back into the source directory.

Project development cycle

ProjectBuilder begins after a project has been created via Workbench. New projects are initialized with the ControlTier base type library. Typically, a developer is extending one or more of these types to manage the specifics of their application's build and deployment processes.

The project development cycle is undertaken in serveral rough steps:

1. Base directory structure created. The ProjectBuilder commands work relative to this base directory.
2. Create and build a type. The next section describes the development cycle for types.
3. Define objects. Each type module can contain files that define type instances.
4. Build the whole project. Process all the modules and build and deploy them.

To begin using ProjectBuilder one must first define an instance of it in the desired project. The user can name the instance anyway they like, but two key bits of information should be specified:

1. basedir: This is the base directory where the modules will reside
2. install-root: This is the directory where the build artifacts will be written

An instance of Project builder can be done either within Workbench or via CTL.

• Via Workbench: Navigate to ProjectBuilder type and create an instance specifying the basedir install-root in the web form.
• Via Antdepo: Run the Register command using the -basedir and -installroot arguments:

  ctl -p project -t ProjectBuilder -o object -c Register -- \
  	  -basedir /path/to/modules -installroot /path/to/target -install

The ProjectBuilder is now ready to use to develop a type.

Type development cycle

Typically, one uses ProjectBuilder to develop a type in three steps:

1. Define the type. Create a new type using the create-type command.

   ctl -p project -t ProjectBuilder -o object -c create-type -- \
   	  -type NewType -supertype SuperType
   
   

   Edit the type.xml file generated by create-type. Define various commands and schema info as desired. Implement command handlers and template files for the type.
2. Build the type. Use the build-type command to process the type.xml and package the type definition and deploy the module.

   ctl -p project -t ProjectBuilder -o object -c build-type -- \
   	  -name NewType -upload -deploy

3. Test a command in your new type. Register it and then run command:

   ctl -p project -t NewType -o object -c Register -- \
   	-basedir /path/to/basedir -installroot /path/to/install -install

   ctl -p project -t NewType -o object -c command
   
   

Preparing for deployment

To prepare for widespread use of the project resources developed using ProjectBuilder, one might consider the following next steps:

• Create a type library: All the types defined by the ProjectBuilder can be packaged together into a library using the build-library command. A type library is a handy way to reuse the type definition in other projects.
• Generate documentation: Type definitions can also be processed to generate a set of Forrest documentation using the generate-forrest-docs command.
• Create object data: Once the types have been suitably developed, one may wish to define an object model based on the type definitions. ProjectBuilder provides the generate-objects command which will bootstrap a set of data files containing instances of the types defined in the type.xml. Those generated files can be used as input for the load-objects command which reads in data as plain text files and loads it into Workbench.
• Create a project build: All types can be checked out, built, packaged and loaded into the repository via the common Builder Build workflow:

  ctl -p project -t ProjectBuilder -o object -c Build

Super Type

Builder

Allowed Child Dependencies

• BuilderBuildFile¹
• BuilderBuildTarget¹
• BuilderScmBinding¹
• BuilderScmConnection¹
• BuilderScmLabel¹
• BuilderScmModule¹
• BuilderStageExtension¹
• BuilderStageFilebase¹
• ProjectBuilderDocBase ¹
• ProjectBuilderForrestHome ¹
• ProjectBuilderOrganizationDescription ¹
• ProjectBuilderOrganizationName ¹
• ProjectBuilderOrganizationURL ¹
• ProjectBuilderProjectDescription ¹
• ProjectBuilderProjectName ¹
• ProjectBuilderProjectURL ¹

¹: These types have a Singleton constraint. Only one instance may be added as a resource.

Allowed Parent Dependencies

• Node

Exported Attributes

Defaults for Imported Attributes

Command Overview

Given the number of commands available with ProjectBuilder, it may be difficult to get an overall view of what each command can do. The following table groups the commands into several categories and provides a link to the command's documentation.

build-library

Builds a library

Generates a library jar file from the types found in the -basedir, saving the jar file into the -targetdir. The file created is named "library-${opts.name}.jar".

The builddir should contain types like this:

• ${opts.basedir}/
  • Type1/
    • type.xml
    • ...
  • Type2/
    • type.xml
    • ...

Usage

build-library [-basedir <>] [-name <common>] [-targetdir <>]

Options

build-type

Builds and packages a type defined in the type.xml

Reads the ${opts.basedir}/${opts.type}/type.xml and generates a set of files suitable for uploading to the server. The type.xml and generated files are packaged into a jar along with any other content found within the module directory hierarchy. After build-type has been run for the first time you will notice two new files: type.rdf and module.rdf. These files contain the same data as found in the type.xml file but use the RDF format, the native representation Workbench uses to maintain the model.

Pre-/post-build hooks

The build-type uses an internal build script that supports two optional Ant build files in the top level of the type directory structure.

• pre-build.xml: Optional ant file to run before the build.
• post-build.xml: Optional ant file to run after the build.

Interaction with Workbench

To assist the development cycle, the -upload and -deploy flags will also upload the the resulting jar and load into the Workbench server, then deploy it to the working CTL context.

	    ad -p project -t ProjectBuilder -o name -c build-type -- \
	    	  -type ${opts.type} -upload -deploy
	  

As Workbench reads in the contents of the jar, it will generate command implementations if they are not found in the jar. This process works pretty well for workflows and shell command types. Besides expanding type.xml command definitions into implementation files, Workbench also generates any needed CTL properties files.

Usage

build-type [-basedir <>] [-deploy] [-targetdir <>] -type <> [-upload]

Options

convert-rdf

Convert type and module rdf to type.xml

Workbench uses RDF as its model representation format and serializes model data into two RDF/XML files: type.rdf and module.rdf. This utiltity command will convert type and module RDF files into the type.xml format.

If the -basedir and -type options are specified, then the the type.xml file is generated into that directory, and the module and type rdf are looked for in there as well. First "type.rdf" and "module.rdf" are looked for, then "type.n3" and "module.n3".

If basedir is not specified, then all three options modulerdf, typerdf, and output must be specified.

rdfformat defaults to "RDF/XML", but if basedir is specified, "N3" will be used if "type.n3" and "module.n3" exist and the "*.rdf" files do not.

Examples

Most args of convert-rdf can be left out if the RDF is in the basedir/type directory. If TypeX contained a type.rdf and module.rdf file but no type.xml, the following file structure would be seen:

• TypeX/
  • type.rdf ;# type definition file
  • module.rdf ;# command definition file
  • commands/
  • objects/
  • templates/

The command would just need the -type option:

	  ad -p project -t ProjectBuilder -o name -c convert-rdf -- -type TypeX
	

Usage

convert-rdf [-basedir <>] [-modulerdf <>] [-output <>] [-rdfformat <RDF/XML>] [-removerdf] [-type <>] [-typerdf <>]

Options

create-type

Creates a new type directory and initial type.xml

New types can be added to the project using the create-type command. This seeds the new type from a set of boiler plate files, and creates skelaton module structure.

The module structure includes a set of standard files (like type.xml) and subdirectories as shown below:

• TypeX/
  • type.xml ;# type definition file
  • commands/
  • • commandA.xml ;# implementation for commandA
    • commandB.xml ;# implementation for commandB
  • objects/ ;# holds optional instance data files
  • templates/ ;# holds optional template files

Type definition falls into two areas:

1. Commands: Typically, the primary aspect of the type definiton is the declaration of commands. The process begins by defining a new new command element in the type.xml. Second, a corresponding command implementation is developed. As noted elsewhere, this second step can be left to Workbench if the command is shellscript or workflow type.
2. Schema: The initial type.xml created by create-type includes the basics, declaring the supertype and some boiler plate elements. One might also like to define Settings subtypes and attributes to build up a data model useful to your commands.

Example: Create a type called MyBuilder, a subtype of Builder:

ctl -p project -t ProjectBuilder -o object -c create-type --\
	  -basetype Builder -type MyBuilder
	  

The create-type command will work interactively if no arguments are given.

Usage

create-type [-basedir <>] [-basetype <Deployment>] [-description <>] [-supertype <>] [-type <>]

Options

find-objects

Queries project for objects matching query params

This command is used to query the active model on the server for any objects matching the option specifications and saves the result in a session that can be used by purge-objects.

Usage

find-objects [-name <.*>] [-purge] [-session <find.session>] [-type <Deployment>]

Options

generate-forrest-docs

generate forrest docs from type.xml files

This command creates a file "type_NAME.xml" for each module "NAME" found in the moduledir. It also creates a file "type_NAME.xml.link" containing links to the type and its secondary types. When generating this xml, any type linked in the doc which exists either as a "type_*.xml" file in the targetdir, or is a named module in the moduledir will have a valid link generated for it.

The basedir is expected to contain:

• ${opts.basedir}/
  • Type1/
    • type.xml
    • ...
  • Type2/
    • type.xml
    • ...

The forrest docs have a standard structure, but any element in the source type.xml file can have additional documentation by including a <doc> tag as an immediate child.

<doc> tags in the type.xml files can contain valid forrest xml fragments, with the addition of two special tags:

• <typelink>TYPENAME</typelink> - produces a link to a type if valid, or the type name otherwise.
• <commandlink>COMMANDNAME</commandlink> - produces a link to the command of the same name of for the current type.

For example, "<typelink>ProjectBuilder</typelink>" produces this: ProjectBuilder, and "<commandlink>generate-forrest-docs</commandlink>" produces this: generate-forrest-docs.

An invalid type link "<typelink>NotAType</typelink>", produces this: NotAType, similarly "<commandlink>invalid-command-name</commandlink>" produces invalid-command-name.

When finished with all the modules, it generates a file "index.xml" which concatenates all "*.link" files found in the targetdir to create a type index. This means you can run this command multiple times with different basedir and the same targetdir, and the index will contain the links to all types used.

If you're reading this, then this command was used to generate this documentation. This text is included as a <doc> tag inside the <command> tag:

	    <command name="generate-forrest-docs" description="generate forrest docs from type.xml files"
	    command-type="AntCommand">
	    <doc>
	    ...
	    </doc>
	    ...
	    </command>
	  

The <doc> tag must contain only valid forrest document content xml. (See the DTD.) Any elements/tags that are not recognized by forrest will cause an error when generating documentation from the xml file.

The xml inside the <doc> tag will be inserted into the forrest output document in one of two locations: directly underneath a <section> tag or a <td> tag (in some cases).

This means that for the most part you will need to include <p>...</p> tags around your documentation content. Also note that tags like <ol>, <ul>, <table> cannot be placed inside <p> tags, as that is not valid forrest content.

The XSL transformation used by this command tries to be smart about the content of the <doc>tag. If it looks like it is just textual content with normal text markup, then the content is wrapped in <p>...</p> automatically. Otherwise it is included as-is in the forrest xml output, so you must be sure it is valid.

Example: this is invalid:

	    <doc>
	    This is some text, and some <em>emphasis</em>.
	    <p>
	    Here is a "&lt;p&gt;" embedded.
	    </p>
	    </doc>
	  

It must be changed to this:

	    <doc>
	    <p>
	    This is some text, and some <em>emphasis</em>.
	    </p>
	    <p>
	    Here is a "&lt;p&gt;" embedded.
	    </p>
	    </doc>
	  

Whereas this:

	    <doc>
	    This is some text, and some <em>emphasis</em>.
	    </doc>
	  

Will be wrapped as a valid paragraph in the output:

	    <document>
	    ...
	    <p>
	    This is some text, and some <em>emphasis</em>.
	    </p>
	    ...
	    </document>
	  

Usage

generate-forrest-docs [-basedir <>] [-docbase <>] [-forresthome <${env.FORREST_HOME}>] [-overwrite] [-targetdir <${opts.docbase}/build>]

Options

generate-objects

Generates an initial set of object data files from the type definition.

Reads type definitions and generates an initial set of object data files suitable to load in the server with load-objects.

Example: Bootstrap files containing deployment and setting object info

ctl -p project -t ProjectBuilder -o object -c generate-objects --\
	     -type Type
	  

Files will be written to directory: ${opts.basedir}/${opts.type}/objects

Usage

generate-objects [-basedir <>] [-basetype <all>] [-delimiter <:>] [-format <tabular>] [-load] [-name <default>] [-overwrite] [-type <>]

Options

load-objects

Loads object data from a file into the project model on the server

The load-objects command provides the capability to read type instance data from a file and load it into the active project model running on the server.

Each type's module directory can contain an objects sub directory where files containing object data can be stored.

The example below shows setting data being loaded on to the server:

ctl -p project -t ProjectBuilder -o object -c load-objects --\
	  -type TypeX -filename setting.tab -basetype setting
	  

Load-objects can default option values depending on other existing args. For example, if the file base is called setting the resource type will automatically be set to that. The reverse is also true. For example:

	    ad -p project -t ProjectBuilder -o object -c load-objects --\
	       -type TypeX -filename setting.tab 
	    vs.
	    ad -p project -t ProjectBuilder -o object -c load-objects --\
	       -type TypeX -basetype setting
	  

If all object data files should be processed, one can leave out both options:

ctl -p project -t ProjectBuilder -o object -c load-objects -- -type TypeX

Tabular format

The tabular format is a delimited data format that has columns separated by the -delimiter argument value and records separated by newlines. Any character string can be used as a delimiter but the : (colon) character is default. Currently, there are two kinds of 'table' files used to declare instances of Deployment and Setting subtypes. A third can be used to declare object to object dependencies.

An example set of .tab files in a module directory structure are shown below:

• TypeX/
  • type.xml
  • commands/
  • objects/
  • • binding.tab
    • deployment.tab
    • setting.tab
  • templates/

Each format uses a different set of columns described as follows:

		  # deployment.tab format -
		  # type:name:install-root:basedir:rank:description
		  Apache:staging:/usr/local/apache2:/var/apache:3:the staging instance	  
		

ctl -p project -t ProjectBuilder -o object -c load-objects -- \
	  -type TypeX -filename deployment.tab  -basetype deployment

	    # setting.tab format -
	    # type:name:settingValue:settingType:description
	    ApacheDocroot:commonDocroot:/docroot:documentRoot:web docroot
	  

ctl -p project -t ProjectBuilder -o object -c load-objects -- \
	  -type TypeX -filename binding.tab  -basetype setting

	    # binding.tab format -
	    # parentType,parentName:childType,childName
	    # Apache -> ApacheSetting
	    Apache,staging:ApacheDocroot,commonDocroot
	  

ctl -p project -t ProjectBuilder -o object -c load-objects -- \
	  -type TypeX -filename binding.tab  -basetype binding

Usage

load-objects [-basedir <>] [-basetype <>] [-delimiter <:>] [-filename <>] [-format <tabular>] [-type <>]

Options

purge-objects

Deletes objects from the project model on the server

Purge-objects reads input generated by find-objects and for each object listed in the specifed session, removes it from the active model on the server.

If the ignoreparents flag is specified, then the object will be removed even if it is a child dependency of another object.

Usage

purge-objects [-ignoreparents] [-name <.*>] [-session <find.session>] [-type <Deployment>]

Options

refactor-rename

Replace text within matching files of a directory

If project option is specified, then only files that appear to be Jena models for that project name will be modified. (Having the file name pattern "*_U${opts.project}"). Otherwise, any files with extensions as listed in the exts option (sans-periods) will be modified.

Usage

refactor-rename [-basedir <>] [-exts <.xml,.properties,.txt,.rdf,.html,.n3,.conf>] -pattern <> [-project <>] -replace <>

Options

repoImport

Uploads type module jar files to the repository

Searches the targetdir for files matching "type-version.extension". If found, then the file is uploaded as a type jar. This command uses the module-uploadjar Ant task to interact with the server to load the type jar into the active model for the project. Workbench will perform file generation for shellscript and workflow command handlers, if they were not present in the model.

Usage

repoImport [-buildstamp <>] [-extension <jar>] [-filebase <>] [-propfile <builder.properties>] [-targetdir <>] [-version <[0-9]+>]

Options

runBuildScript

runs the project build script

This command invokes the specified buildfile to execute the project build cycle.

Default build.xml

By default, ProjectBuilder uses the Ant build file: ProjectBuilder/lib/build.xml. This build.xml provides a set of targets:

Users wishing to extend or implement their own project build cycle can develop a build.xml of their own and create an object dependency to an instance of BuilderBuildFile that contains the path to the file.

Likewise, user specific build targets can be specified using an instance of BuilderBuildTarget.

Usage

runBuildScript [-basedir <>] [-buildfile <>] [-buildstamp <>] [-buildtarget <>] [-targetdir <>]

Options

The following types are defined for use with ProjectBuilder.

ProjectBuilderSetting

Overview

ProjectBuilderSetting: An abstract ProjectBuilder setting

Design

Super Type

Setting

Constraints

Allowed Parent Dependencies

• ProjectBuilder

ProjectBuilderForrestHome

Overview

ProjectBuilderForrestHome: Path to FORREST_HOME directory

Design

Super Type

Setting

Constraints

Allowed Parent Dependencies

• ProjectBuilder

Attributes

Exported Attributes

ProjectBuilderProjectName

Overview

ProjectBuilderProjectName: The project name

Design

Super Type

Setting

Constraints

Allowed Parent Dependencies

• ProjectBuilder

Attributes

Exported Attributes

ProjectBuilderProjectDescription

Overview

ProjectBuilderProjectDescription: Brief description about the purpose of the project

Design

Super Type

Setting

Constraints

Allowed Parent Dependencies

• ProjectBuilder

Attributes

Exported Attributes

ProjectBuilderOrganizationName

Overview

ProjectBuilderOrganizationName: Organization responsible for the project

Design

Super Type

Setting

Constraints

Allowed Parent Dependencies

• ProjectBuilder

Attributes

Exported Attributes

ProjectBuilderOrganizationDescription

Overview

ProjectBuilderOrganizationDescription: Descrpiton of organization responsible for the project

Design

Super Type

Setting

Constraints

Allowed Parent Dependencies

• ProjectBuilder

Attributes

Exported Attributes

ProjectBuilderOrganizationURL

Overview

ProjectBuilderOrganizationURL: URL to the organization home page

Design

Super Type

Setting

Constraints

Allowed Parent Dependencies

• ProjectBuilder

Attributes

Exported Attributes

ProjectBuilderProjectURL

Overview

ProjectBuilderProjectURL: URL to the project home page

Design

Super Type

Setting

Constraints

Allowed Parent Dependencies

• ProjectBuilder

Attributes

Exported Attributes

ProjectBuilderDocBase

Overview

ProjectBuilderDocBase: Path to documentation directory

Design

Super Type

Setting

Constraints

Allowed Parent Dependencies

• ProjectBuilder

Attributes

Exported Attributes