Jenkins, friend or foe 2/2

Although installation of Jenkins is relatively easy, configuration is not if you never done this before. The problem is that 'everything'  is code and I wanted to make use of multibranche pipelines. However, a lot of Jenkins examples on the Internet refer to using the Jenkins GUI. I do not want to use the GUI. It must be possible to reinstall everything in one click and I don't want to do any manual task.

Configuration

Most of the configuration is done by means of 'hook' scripts. There are different types of hooks and methods (see Groovy Hook Script), but for most configuration items the 'init hook', using .groovy scripts is sufficient.

In the Ansible installation mentioned in the previous post, the installation directory of Jenkins is /var/lib/jenkins/, so the init scripts are located in.

/var/lib/jenkins/init.groovy.d/*.groovy

The scripts are executed in lexical order, so make sure to use a file naming convention. I use [A..Z]_init*.groovy, but you could also use a numbering.

A_init_git.groovy

Jenkins must be able to connect to a Git repository and does this by means of SSH. This means you need to generate an SSH key pair; I won't go into details how to generate this. Just Google and you find some examples.

The private key must be stored as credential in Jenkins and is used later in the script that creates the Jenkins jobs. Of course, you can insert the private key manually in Jenkins, but we want to automate everything, so also this step is done by means of scripting. First step in the process is to generate the key pair on the Jenkins server (not in scope of this blog post). The private key in this example is located at '/var/lib/jenkins/ssh_git'. As part of the Jenkins (re)start, the A_init_git.groovy script is executed and the key is installed. The script looks like this:

import jenkins.model.*
import hudson.security.*
import hudson.tasks.*
import jenkins.plugins.git.*
import com.cloudbees.plugins.credentials.*;
import com.cloudbees.plugins.credentials.common.*
import com.cloudbees.plugins.credentials.domains.*
import com.cloudbees.plugins.credentials.impl.*
import com.cloudbees.jenkins.plugins.sshcredentials.impl.*
import hudson.plugins.sshslaves.*;

println "==> Executing A_init_git.groovy"

String keyFile = "/var/lib/jenkins/ssh_git"
String keyID = "ssh_git"
String keyPassphrase = ""
String keyUsername = "ssh_git"
String keyDescription = "Read access to repositories"

// Get the credentials provider
def global_domain = Domain.global()
def credentials_store = Jenkins.instance.getExtensionList('com.cloudbees.plugins.credentials.SystemCredentialsProvider')[0].getStore()

println "--> Insert SSH credentials for accessing Git via SSH"
credentials = new BasicSSHUserPrivateKey(
CredentialsScope.GLOBAL,
keyID,
keyUsername,
new BasicSSHUserPrivateKey.FileOnMasterPrivateKeySource(keyFile),
keyPassphrase,
keyDescription
)
credentials_store.addCredentials(global_domain, credentials)

println "==> End A_init_git.groovy"

B_init_credentials.groovy

One of the other things that need to be done first is setting up the credentials (users + password) in Jenkins, so they can be used in the other init scripts. To reduce manual work this can be done automated and in a secure way. One approach is:

• Create a yml file with username + password; this is a plain file. This file is maintained in a secure environment (e.g. Keypass). The format (used in this example) is:

- env: all
  username_1: password_1
  username_2: password_2

• Encrypt it manually with an AES 128 bit algorithm, using a passphrase; the encrypted file is stored in Git (there is example code on the Internet that shows how to perform the encryption/decryption)
• The passphrase is given as 'extra variable' during startup of the playbook, or - even better -  it is configured by means of Ansible Vault
• One of the steps in the Ansible playbook is to copy the encrypted yml file from Git to the Jenkins server
• The second Ansible step is to decrypt the yml file using the passphrase
• Third Ansible step is to restart Jenkins; this processes the *.groovy files, including the B_init_credentials.groovy
• The step after a Jenkins restart is to delete the decrypted file; also in case something went wrong, the decrypted file must always be deleted. 
• The groovy file itself looks something like this:

import jenkins.model.*
import hudson.security.*
import hudson.tasks.*
import com.cloudbees.plugins.credentials.*;
import com.cloudbees.plugins.credentials.common.*
import com.cloudbees.plugins.credentials.domains.*
import com.cloudbees.plugins.credentials.impl.*

println "==> Executing B_init_credentials.groovy"

// Read the credentials file
CredentialsUtils utils = new CredentialsUtils()
def map = [:]
// Read the credentials file, decrypted by Ansible
String propertyFile = 'credentials_decrypted.yml'
map = utils.getPropertyMap ('all', propertyFile)

// Get the credentials provider
def global_domain = Domain.global()
def credentials_store = Jenkins.instance.getExtensionList('com.cloudbees.plugins.credentials.SystemCredentialsProvider')[0].getStore()

if (map['username_1'] != "") {
  Credentials credentialsUser_1 = new UsernamePasswordCredentialsImpl (CredentialsScope.GLOBAL, "user_1", "", map['username_1'], map['password_1'])
  credentials_store.addCredentials(global_domain, credentialsUser_1)
}

if (map['username_2'] != "") {
  Credentials credentialsUser_2 = new UsernamePasswordCredentialsImpl (CredentialsScope.GLOBAL, "user_2", "", map['username_2'], map['password_2'])
  credentials_store.addCredentials(global_domain, credentialsUser_2)
}

println "==> End B_init_credentials.groovy"

class CredentialsUtils {
  /* Returns a map of all properties in the file 'fileName'
  */
  def getPropertyMap (def target, def fileName) {
    // Read the properties and build a property map
    def propertyfile = new File(fileName)
    String propertyContent = propertyfile.text
    boolean sectionFound = false
    def propertyMap = [:]
    String key = ""
    String value = ""
    def lines = propertyContent.readLines()
    lines.each { 
      line ->
      key = getKeyValueFromPropertyLine(line, true)
      if (key == "- env")
      {
        value = getKeyValueFromPropertyLine(line, false)
        if (value.toUpperCase() == target.toUpperCase())
          sectionFound = true
        else
          sectionFound = false
      }
      else if (sectionFound) {
        key = getKeyValueFromPropertyLine(line, true)
        value = getKeyValueFromPropertyLine(line, false)
        propertyMap[key] = value
      }
    }
   return propertyMap
  }

  /* Returns the key- or value part of a line from a property file
  */
  String getKeyValueFromPropertyLine (String line, boolean k) {
    String kv = ""
    try {
      if (k)
        kv = line.substring(0, line.lastIndexOf(":")).trim()
      else
        kv = line.substring(line.lastIndexOf(":") + 1, line.length()).trim()
    }
    catch (e) {
      // Ignore exceptions and just continue
    }
    return kv
  }
}

C_init_ldap.groovy

This script configures the Jenkins ldap plugin and for convenience the password is in plain text (never do this in a real script). The 'tricky' part is, that to find a user the ldap sometimes returns more than one user. The ldap plugin cannot handle this and it results in an exception; basicly this means that you can't login to Jenkins. The 'userSearch' property defined in the script below takes care of returning only one user.

import jenkins.model.*
import hudson.security.*
import org.jenkinsci.plugins.*
import java.util.logging.Logger
import java.util.logging.Level

println "==> Executing C_init_ldap.groovy"

Logger logger = Logger.getLogger("")

String server = 'ldaps://ldap-p.mycompany.com:636'
String rootDN = 'o=mycompany,c=fr'
String userSearchBase = ''
String userSearch = '(&(objectClass=person)(uid={0}))'
String groupSearchBase = ''
String managerDN = 'CN=username,OU=Users,DC=example,DC=org'
String managerPassword = 'secret'
boolean inhibitInferRootDN = false

SecurityRealm ldap_realm = new LDAPSecurityRealm(server, rootDN, userSearchBase, userSearch, groupSearchBase, managerDN, managerPassword, inhibitInferRootDN)
Jenkins.instance.setSecurityRealm(ldap_realm)

Jenkins.instance.save()
logger.info("--> Security set to LDAP " + server)
println "==> End C_init_ldap.groovy"

H_init_jobs.groovy

The Jenkins jobs are also created by means of a script. In this case, a multibranch pipeline job is created. The beauty of multibranch pipeline jobs is that you are able to script the CI/CD workflow (or at least a part), which is of course very flexible. The private SSH key, stored as Jenkins credential in A_init_git.groovy, is used again in this script.

Note, that the script also creates lockable resources. A lockable resource is a resource - often a test environment - that is allocated during execution of a job. If another job would be started, it allocates another resource until all resources are busy. This queues new jobs until a resource is released again. An example of lockable resources will be given in another blog post that covers the multibranch pipeline job itself.

import jenkins.model.*
import jenkins.plugins.git.*
import hudson.tasks.*
import jenkins.branch.*
import org.jenkinsci.plugins.workflow.multibranch.*
import org.jenkins.plugins.lockableresources.LockableResourcesManager;

println "==> Executing H_init_jobs.groovy"

def instance = Jenkins.getInstance()
String keyID = "ssh_git"

try {
  // Configures Multi Branch project using the ssh key credentials
  WorkflowMultiBranchProject mp = instance.createProject(WorkflowMultiBranchProject.class, "MyProject")
  mp.getSourcesList().add(new BranchSource(
    new GitSCMSource(null, "ssh://git.mycompany.com/myproject.git", keyID, "*", "", false),
    new DefaultBranchPropertyStrategy(new BranchProperty[0])))
}
catch(err) {
  println 'Job already exists; continue...'
}

// Setup test environments
LockableResourcesManager mgr = LockableResourcesManager.class.get();
mgr.createResourceWithLabel("T1", "TestEnvironment")
mgr.createResourceWithLabel("T2", "TestEnvironment")

println "==> End H_init_jobs.groovy"

These are just a few examples of init scripts. Of course you can do much more, but that is beyond the scope of this post.