Fulgens

This software helps with building, deploying and running software locally. It is a shell script generator to orchestrate software components locally.

Fulgens also documents version compatibility for your software's dependencies like databases, Tomcat and Java.

It is not a tool to build software. It is not a tool to remotely deploy software. It is generally not a DevOps tool by any means.

The tool was made for software developers to document deployment dependencies like runtime environments and databases, as well as running it locally as easy as possible.

It standardizes "run this software locally", like maven standardized "build this software".

How to use it?

1.) Install via npm: npm -g install fulgens

2.) Create a Fulgensfile (there is an example down below)

3.) Execute fulgens in the same directory as a Fulgensfile and generate a bash script you can use to build, deploy and run your software locally. Start with fulgens | bash -s -- -h to read the help for your generated bash script.

Alternatively you can fulgens > run_local.sh to persist the generated bash script onto the filesystem.

If you've shfmt in your path fulgens will format the generated output via shfmt automatically.

A simple example / Tutorial

Create a Web project via: mvn archetype:generate -DgroupId=de.oglimmer -DartifactId=MyApp -DarchetypeArtifactId=maven-archetype-webapp -DinteractiveMode=false. Step into the new project directory and create a minimal Fulgensfile with this content there:

module.exports = {

  config: {
    SchemaVersion: "1.0.0",
    Name: "example",
  },

  software: {
    "MyApp": {
      Source: "mvn",
      Artifact: "target/MyApp.war"
    },

   tomcat: {
      Source: "tomcat",
      Deploy: "MyApp"
    }
  }
}

Run fulgens>run_local.sh to create a bash script. Follow with chmod 755 run_local.sh to make it runnable and finally build, deploy and run via ./run_local.sh -f.

Now browse to http://localhost:8080/MyApp/

Now try ./run_local.sh -f -t tomcat:docker to run the Tomcat inside a Docker container. Or ./run_local.sh -f -b docker to build inside Docker, but run on your local machine.

Also checkout ./run_local.sh -h to see all available options.

Supported software

A Fulgensfile defines all software components needed to run a project. Supported software is

• maven (build)
• java (start)
• node (start)
• shell script (start)
• tomcat (host war files)
• mysql (database)
• couchdb (database)
• redis (database)

Futhermore Vagrant/VirtualBox is supported as well.

A Fulgensfile and its anatomy

A Fulgensfile may have the file extension ".js", it is recommended to name it "Fulgensfile.js".

A json-schema.org compliant json-schema file can be found here

The Fulgensfile is JavaScript file defining an (JSON) object and assigning it to module.exports.

It must contain two attributes

• config
• software

and it may contain a third attribute versions.

This is a minimal and basic Fulgensfile:

module.exports = {
  config: {
    SchemaVersion: "1.0.0",
    Name: ".....",
  },
  software: {
  }
}

The root object config

config defines overall configuration parameters.

Attribute name	Type	Description
SchemaVersion	string	Required. Must be "1.0.0"
Name	string	Required. The name of the software
Vagrant	object	Defines how a vagrant VM should be spun up
UseHomeM2	boolean	Default is false, if set to true ~/.m2 will be used for Vagrant and Docker environments
BuildDependencies	object	Defines additional apt or npm packages needed for a build
Dependencycheck	array of strings	shell code to execute to check if a dependency or prerequisite is ok

The object Vagrant

Must be inside the root object config.

Attribute name	Type	Description
Box	string	Required. Vagrant box name like ubuntu/xenial64. Only Debian based systems are supported.
Install	string	Space separated apt packages to install. E.g. "maven openjdk-8-jdk-headless docker.io"
BeforeInstall	array of strings	shell command to execute before the apt packages from Install are installed
AfterInstall	array of string	shell command to execute after the apt packages from Install are installed

The object BuildDependencies

Must be inside the root object config.

Attribute name	Type	Description
Apt	array of strings	Apt package names.
Npm	array of strings	Npm package names. The apt package "nodejs" will be automatically added to apt if this is array as at least one element

The root object versions

This section is optional and can be used to define and document version compatiblity.

Attribute name	Type	Description
Docker	string	Tag of a docker image. 'latest' is always used unless this attribute is defined for a software compoenent
JavaLocal	string	Works only on macOS. Sets JAVA_HOME to this version for this component
Download	string	Overrides the download version
TestedWith	string	A description to document tested versions
KnownMax	string	A description to document a maximum version

The root object software

All software components will be configured under software, by adding an attribute for each component like a mysql, doing a maven build or starting a tomcat.

There are no other attributes than software components. The attribute name is the software component name and can be freely choosen. The value is an object defining the software component. The only always required attribute per software component is "Source" which defines the type of software and thus plugin to be used.

Attribute name	Type	Description
Source	strings	"java", "mvn", "node", "mysql", "couchdb", "redis", "shell"

Defining a maven software component

To build a Java project via maven, the maven plugin can be used to build locally or within a docker container. When doing a local build the -j JAVA_VERSION setting is respected.

Required and mvn specific attributes:

Attribute name	Type	Description
Source	string	Required. Must be "mvn"
Mvn	object	Optional. Defines maven specific config
Mvn.Goal	string	Optional. Default is "package". Defines a maven goal, e.g. "install"

Supported common attributes:

Git, Param, Dir, dockerImage, EnvVars, BeforeBuild, AfterBuild

Example software component for mvn:

software: {
    foo: {
        Source: "mvn",
        Artifact: "target/foo.war",	
    }
}

Defining a nodejs software component

To run a node.js application a local node binary can be used, it is also possible to start and run a Node application inside a docker container.

Required and node specific attributes:

Config param	Type	Description
Source	string	Required. Must be "node"
Start	string	Required. Path to entry point javascript file

Supported common attributes:

Git, Param, Dir, dockerImage, EnvVars, BeforeBuild, AfterBuild, BeforeStart, AfterStart, ExposedPort

Example software component for node:

{
software: {
    foo: {
      Source: "node",
      Start: "src/server/start.js"
    }
}

Defining a java software component

A Java program can be started on the local machine or inside a docker container.

Required and java specific attributes:

Config param	Type	Description
Source	string	Required. Must be "java"
Start	string	Required. Name of another software component which defined an Artifact attribute, as this will be started

Supported common attributes:

Git, Param, Dir, dockerImage, EnvVars, BeforeStart, AfterStart, ExposedPort

Example software component for java:

software: {
  javaprogram: {
    Source: "java",
     Start: "<<name of another software component>>"
  }
}

Defining a shell software component

A shell script can be started on the local machine or inside a docker container.

Required and shell specific attributes:

Config param	Type	Description
Source	string	Required. Must be "shell"
Start	string	Required. Name of another software component which defined an Artifact attribute, as this will be started

Supported common attributes:

Git, Param, Dir, dockerImage, EnvVars, BeforeStart, AfterStart, ExposedPort

Example software component for shell:

software: {
  program: {
    Source: "shell",
     Start: "<<name of another software component>>"
  }
}

Defining a tomcat to host a software component

To run a web application a fresh Tomcat can be downloaded, extracted and started, it is also possible to start and run a Tomcat instance inside a docker container. Also an existing local Tomcat installation can be reused.

Required and tomcat specific attributes:

Config param	Type	Description
Source	string	Required. Must be "tomcat"
Deploy	string	Required. Name of another component. The file defined in its Artifact config will be copied to Tomcat's /webapp folder
Lib	array of strings	Name of another component. The file defined in its Artifact config will be copied to Tomcat's /lib folder
SourceTypes	array of strings	Optional. Default is "download", "local", "docker". This parameter can limit the option of source types.

Supported common attributes:

Git, Param, Dir, dockerImage, EnvVars, BeforeStart, AfterStart, ExposedPort

Example software component for tomcat:

software: {
  tomcatWebServer: {
    Source: "tomcat",
    Deploy: "<<name of a different software compoenent>>",
    Lib: [ "<<name of a different software compoenent>>" ],
    SourceTypes: [ "download", "local" ]
  }
}

Defining a mysql database

A MySQL database can be started and used inside a docker container or an existing local installation can be reused.

Required and mysql specific attributes:

Config param	Type	Description
Source	string	Required. Must be "mysql"
Mysql	object	optional, defines the mysql specific config
Mysql.Schema	string	Creates this schema if it doesn't exist
Mysql.Create	array of string	sql files to execute after startup
Mysql.RootPassword	string	Root password to set

Supported common attributes:

Git, Param, Dir, dockerImage, EnvVars, BeforeStart, AfterStart, ExposedPort

Example software component for mysql:

software: {
  "mysqlserver": {
    Source: "mysql",
    Mysql: {
      Schema: "my_new_schema",
      Create: [
        "src/mysql/initial_ddl.sql",
        "src/mysql/initial_data.sql"
     ],
     RootPassword: "root"
    }
}

Defining a couchdb database

A CouchDB database can be started and used inside a docker container or an existing local installation can be reused.

Required and couchdb specific attributes:

Config param	Type	Description
Source	string	Required. Must be "couchdb"
CouchDB	array of objects	optional, defines the couchdb specific config
CouchDB.Schema	string	Creates this schema if it doesn't exist
CouchDB.Create	array of string	json file imported after startup, usually something like views

Supported common attributes:

Git, Param, Dir, dockerImage, EnvVars, BeforeStart, AfterStart, ExposedPort

Example software component for couchdb:

software: {
  'couchdbserver': {
    Source: "couchdb",
    CouchDB: [{
      Schema: "toldyouso",
      Create: [ "src/couchdb/_design-User-view.json" ]
    }]
  }   
}

Defining a redis database

A Redis database can be started and used inside a docker container or an existing local installation can be reused.

Required and redis specific attributes:

Config param	Type	Description
Source	string	Required. Must be "redis"

Supported common attributes:

Git, Param, Dir, dockerImage, EnvVars, BeforeStart, AfterStart, ExposedPort

Example software component for redis:

software: {
  'redisserver': {
    Source: "redis"
  }   
}

Common attributes

Many attributes are supported by multiple types of software components.

This spreadsheet shows which plugins support which parameters: Plugin Parameter overview

Git

Fulgens can clone a git repository.

Config param	Type	Description
Git	string	git URL to clone or pull. A branch name can be added after a space.

Param

A software component can be made optional. If done so a parameter must be given to the generated bash script to run the code for this software component.

Config param	Type	Description
Param	object	Defines a script switch / parameter to make this software component optinal
Param.Char	string with size 1	character to use for this script parameter. Must not use h,s,c,k,t
Param.VariableName	string	Shell variable name to set to YES if the parameter was given to the script
Param.Description	string	A description for the help section of the script

Example:

software: {
  barDependency: {
    Source: "mvn",
    Git: "https://github.com/foo/bar.git",
    Dir: "$$TMP$$/bar-lib",
    Mvn: {
      Goal: "install"
    },
    Param: {
      Char: "o",
      VariableName: "BAR_LIB_ENABLED",
      Description: "Clone, build and install bar dependency"
    }
  }

Dir

A software component uses the same directory than the Fulgensfile by default. However the working directory can be changed by the attribute Dir.

Config param	Type	Description
Dir	string	Defines the directory where the build will performed. Must be inline where the source code be found. Default is "."

BeforeStart and AfterStart

It is possible to execute shell code before or after a software component has started.

Config param	Type	Description
BeforeBuild	array of string	Bash code to execute before the component is started
AfterBuild	array of string	Bash code to execute after the component is started

DockerImage & DockerMemory

Each software component type (plugin) has a docker image associated. This can be changed via this attribute, also it is possible to set the max memory parameter for the docker container.

Config param	Type	Description
DockerImage	string	docker image name without version
DockerMemory	string	max memory parameter "-m" for docker

EnvVars

This attributes allows the definition of environment variables. They will be passed to the application in an appropriate manner.

Config param	Type	Description
EnvVars	array of objects	object with Name and Value
EnvVars.Name	string	string which defines the name of the environment variable
EnvVars.Value	string	string which defines the value of the environment variable. $$TMP$$ will be replaced with the working directory of the generated file. $$VALUE$$ will be replaced with the hostname of the Source
EnvVars.Source	string	string which defines the source component
EnvVars.DockerOnly	boolean	if true will be set for Docker only

ExposedPort

When starter a docker container the system needs to know which port a component exposes.

Config param	Type	Description
ExposedPort	number	The TCP port number this software bind to

BeforeBuild and AfterBuild

It is possible to execute shell code before or after a build (mvn/node) is done.

Config param	Type	Description
BeforeBuild	array of string	Bash code to execute before the build is performed
AfterBuild	array of string	Bash code to execute after the build is performed

Config files

In Fulgens a software components can define config files. This mechanism allows it to override or add static configuration values, as well as to replace host names in configuration files at run-time.

Config param	Type	Description
Name	string	Required. Name of the config file
Content	array of objects	Optional. Defines static or dynamic content added or replaced in the config file. Dynamic parts are set at run-time with host names of other resouces
Content.Line	string	Required. Content to add or replace in the config file. It may containt $$VALUE$$ which will be replaced by the host name at run-time given my the Source attribute
Content.Regexp	string	Optional. A regular expression to search in the config file. The first line matching this regular expression will be replaced by the value giving via Line
Content.Source	string	Optional. This references another software component by its name
LoadDefaultContent	string	Optional. Absolute or relative file path (to project root) to a config file. Supports $$TMP$$
AttachAsEnvVar	array of 2 strings	Mutually exclusive to AttachIntoDocker or AttachIntoDockerAsFile. The config file will be attached to the application via an environment variable. The first string defines the name of the environment variable. The second string defines the value where $$SELF_NAME$$ will be replaced by the file path to the config file at run-time
AttachIntoDocker	string	Mutually exclusive to AttachAsEnvVar or AttachIntoDockerAsFile. The config file will be mounted into docker via a directory. This defines the absolute path on the docker filesystem
AttachIntoDockerAsFile	string	Mutually exclusive to AttachAsEnvVar or AttachIntoDocker. The config file will be mounted into docker via a file. This defines the absolute path on the docker filesystem to the config file.

Example:

config: {
    Name: "app.properties",
    Content: [{
      Source:"mysql",
      Line: "mysql.url=jdbc:mysql://$$VALUE$$:3306/schema"
    }, {
      Line: "bind=0.0.0.0"
    }, {
      Line: "user=foobar"
    }],
    AttachAsEnvVar: ["JAVA_OPTS", "-Dapp.filename=$$SELF_NAME$$"]        
    // or
    AttachIntoDocker: "/usr/local/etc/app/local.d"
}

Example files Fulgensfile.js

Many projects in oglimmer's github provide a Fulgensfile. Here is a list of real-world Fulgensfiles:

dsa-talent-calc-web

# Maven, dependency-build, Tomcat

A Java based web application which has a dependency to a different project (which doesn't live inan official maven repository).

See this Fulgensfile.js how to clone and build the dependency before building the Java web applicaton and starting it in a local Tomcat.

deckbuilder for swlcg and swccg

# Maven, Couchdb

Two very similar projects and their Fulgensfiles Fulgensfile.js, Fulgensfile.js building a Java project, starting an empty Couchdb before starting a Tomcat. These projects have a Dependencycheck added to their config section.

Code Your Restaruant (cyc)

# Maven, Tomcat, Java-Standalone-Backend

This Fulgensfile.js first builds a java project, then starts a Couchdb where it adds 3 views. The project consists of 2 parts, a backend server which is a plain java program and a war file hosted on a Tomcat. Both of them need config files.

Lunchy

# Maven, Mysql, Tomcat

A java web applicaton using a Mysql database. The Fulgensfile.js builds the java project, starts the Mysql, deploys the war file to the Tomcat and starts the Tomcat. To use utf-8 a config file is applied to the Mysql, after the database started the schema, tables and initial data is created via 'mvn -DcreateTables=true process-resources'.

Podcast human syncer

# Node, Redis

A very simple Node application, which uses a Redis database. The Fulgensfile.js just starts Redis and node.

Simple Card Game

# Maven, Tomcat

Most simple Fulgensfile.js. Just building a project and starting it on a Tomcat.

Told You So

# Maven, Couchdb, Tomcat, config-file, before/after Build

This Fulgensfile.js builds a project which needs a different pom.xml for java >= 9. It also startes a Couchdb and Tomcat.

Yet Another Tower-Defnese Game (yatdg)

# Maven, Tomcat

Most simple Fulgensfile.js. Just building a project and starting it on a Tomcat.

Linky

# Maven, Node, Couchdb, config-file, before/after Build, environment variables

This Fulgensfile.js starts with cloning the git repositoy of Lucene, building it and starting Lucene as a standalone java process. The a Couchdb is started and 2 schemas with initial views are created. Finally a node program is started. The Couchdb and Node have their own config files and Node is provided several environment variables.

Citybuilder

# Node, Couchdb, config-file

A Couchdb and a Node program is started in this Fulgensfile.js.

Grid Game One (ggo)

# Maven, Tomcat, Build-Dependency

Very simple Fulgensfile.js. It builds a project and starts it on a Tomcat.