Wednesday, December 13, 2017

Marking Jenkins builds with Badges


We can adjust the status of a Jenkins job to Unstable, Failed or Successful, and we can add handy badges to the builds to signal warnings or errors.  All from the Groovy PostBuild Plugin. 

Jenkins is like your older uncle who helps you get through college: Dependable but a little clunky, stuffy but also necessary.  Turns out uncle is wearing a brightly patterned sports coat at times, in the shape of build badges:



What is also nice is that a Jenkins build can have final status other than failed (red ball) or passed (blue ball).  You may have seen NOT_BUILD (grey ball) and broken your head over UNSTABLE (yellow ball).

Aside: What is unstable supposed to mean, you ask?  In Jenkins' slightly warped view of the world, if the build step is successful (ie. the compiler did its job) but the tests failed, a job is unstable.  In normal agile terms, that is just a plain fail.  To add to the confusion, only certain post-build actions are able to move the status to unstable, so you may rarely encounter this.  But it is a great status to (ab)use if you want to show alerts on your job.

Setting build status and badges

So how can we access all this goodness?  Groovy is the answer here.  Note that I am applying groovy to any style Jenkins jobs, not only pipeline jobs that already groovy-fied.

You want to add the Groovy postbuild plugin to your Jenkins and then create a Groovy post-build action.  Here is some sample code that unconditionally set the build to Unstable.   Run it and you should see a yellow ball and an unstable outcome.
   import hudson.model.*
   def build = manager.build
   build.@result = hudson.model.Result.UNSTABLE
   manager.addErrorBadge("That really wasn't very stable")
   return


There are lots of other ways to do the same, and other things you can do, see the documentation for this plugin.  For example, you can also use 'manager.buildUnstable()' instead of that last line.

The Jenkins interface has an nice 'run in sandbox' checkbox, which you really want to check because it limits the abilities of your Groovy scripts.  However, it didn't work for me:  I basically could not run any Groovy code with this set.  The docs say that all the methods we used above are whitelisted, but apparently things have changed (or my Jenkins version is out of date).

Reading and checking build files


For my problem, I needed to get slightly more fancy and tests a few files in the current job.  Note that it is much easier to test for log lines in the current build, the method 'manager.logContains(REGEXP)' can help you with that.

To test for certain files in the current build directory, you can use:
def build = manager.build
def workspace = manager.getEnvVariable('WORKSPACE')

if (new File(workspace + '/mySpecialFile').exists()) {
   manager.buildUnstable()
   manager.addErrorBadge("Unit tests were not run")
   manager.listener.logger.println("Marked unstable")
else {
   manager.listener.logger.println("All well") }

return

This checks for the presence of 'mySpecialFile' under the project root.  If you build has produced such a file, it will be marked unstable and a log line will be emitted.


Getting a list of all variables available to you

How did I find out about the workspace variable?  I ran this piece of code, modified from one of the comments in the Groovy PostBuild Plugin page:

def build = Thread.currentThread().executable
def buildMap = build.getBuildVariables()
buildMap.keySet().each { var -> 
       manager.listener.logger.println( var + "= " + buildMap.get(var)) }
def envVarsMap = build.parent?.lastBuild.properties.get("envVars")
envVarsMap.keySet().each { var -> 
       manager.listener.logger.println( var + ": " + envVarsMap.get(var)) }

This will show the build variables (parameters to the build) and the environment variables.  My result (slightly redacted; no build variables):

BUILD_DISPLAY_NAME: #29
BUILD_ID: 29
BUILD_NUMBER: 29
BUILD_TAG: jenkins-test-29
BUILD_URL: http://localhost:8080/job/test/29/
CLASSPATH: 
DERBY_HOME: /usr/lib/jvm/java-8-oracle/db
EXECUTOR_NUMBER: 1
HOME: /var/lib/jenkins
HUDSON_HOME: /var/lib/jenkins
HUDSON_SERVER_COOKIE: 7......d
HUDSON_URL: http://localhost:8080/
J2REDIR: /usr/lib/jvm/java-8-oracle/jre
J2SDKDIR: /usr/lib/jvm/java-8-oracle
JAVA_HOME: /usr/lib/jvm/java-8-oracle
JENKINS_HOME: /var/lib/jenkins
JENKINS_SERVER_COOKIE: 7.....d
JENKINS_URL: http://localhost:8080/
JOB_BASE_NAME: test
JOB_DISPLAY_URL: http://localhost:8080/job/test/display/redirect
JOB_NAME: test
JOB_URL: http://localhost:8080/job/test/
LANG: en_US.UTF-8
LOGNAME: jenkins
MAIL: /var/mail/jenkins
MANPATH: /opt/OpenPrinting-Gutenprint/man:/opt/OpenPrinting-Gutenprint/man:/usr/local/man:/usr/local/share/man:/usr/share/man:/usr/lib/jvm/java-8-oracle/man
NLSPATH: /usr/dt/lib/nls/msg/%L/%N.cat
NODE_LABELS: master
NODE_NAME: master
NODE_PATH: /usr/lib/nodejs:/usr/lib/node_modules:/usr/share/javascript
PATH: /opt/OpenPrinting-Gutenprint/sbin:/opt/OpenPrinting-Gutenprint/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/snap/bin:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin
PATH+CHROMEDRIVER: /var/lib/jenkins/tools/chromedriver
PWD: /var/lib/jenkins
QT_QPA_PLATFORMTHEME: appmenu-qt5
RUN_CHANGES_DISPLAY_URL: http://localhost:8080/job/test/29/display/redirect?page=changes
RUN_DISPLAY_URL: http://localhost:8080/job/test/29/display/redirect
SHELL: /bin/bash
SHLVL: 1
USER: jenkins
WORKSPACE: /var/lib/jenkins/workspace/test
XDG_RUNTIME_DIR: /run/user/125
XDG_SESSION_COOKIE: f...........0
XDG_SESSION_ID: c20
XFILESEARCHPATH: /usr/dt/app-defaults/%L/Dt

You can check this output against http://localhost:8080/env-vars.html.



 Hope you can use this in your work with Jenkins!

Thursday, August 31, 2017

A pragmatic Git strategy for fast moving teams

Git is a wonderful tool that is so flexible you can use it in many ways.  With that has come the discussion of how to use it best.  Of course, the answer to depends on your priorities.

Here are the priorities I see often in agile teams delivering software fast:

  1. Members of the team need to work in temporary isolation (read: branches)
  2. Git and git history is a way to communicate your code changes to the rest of your team
  3. You have tests that give you confidence in your final code
  4. Recent Git history is valuable if something goes wrong
  5. When you build something, you ship it asap
Let's apply these principles to our thinking about various parts of a Git strategy:

Feature branches:  A pair (or a soloing dev) should work on a branch (isolation), and branches are best organized by topic.  Feature work should be short lived so it does not diverge.  Merge master back into the feature if in doubt.  
git checkout feature
git merge master

Feature branches can all start with a common prefix (ft_) but they do not have to.   

Release branches:  If we really ship all we have, every time, there should usually be no need for release branches.  Just release master, optionally from a couple of commits ago.   If you are just in the middle of a big change, either do not merge that feature yet or make a once-in-a-lifetime release branch. Tag your release:
git tag release-1.1 
git push origin release-1.1

Merging features:  A feature branch will have detailed commit history on it, which is great if you need to go back to it next week.  But to keep master easy to follow, you squash your work into one commit when it goes to master.  Keep the feature branch around for 4 weeks, then delete it.

You can accomplish this in many ways, this one is easy to understand:
git checkout feature # Use the name of your branch here
git checkout -b temp  
# Solve any conflicts between feature and master:
git rebase master 
## if conflicts occur, fix, git add and git rebase --continue

git checkout master
git pull
git merge --squash temp
git commit 
git branch -D temp

At this point, you feature branch will not have changed and all the work of merging it with master is hidden in one commit on master.  Write a recurring calendar event to delete old branches on the first of each month.

During the 'git rebase master' step, you may have to merge files multiple times.  This is tedious but it makes the individual merges easier.  If you are impatient, skip this step and deal with the conflicts during the merge step. 


If your feature branch warrants two or more commits: It happens, you did two things in one feature.  Proceed as follows:
git checkout feature
git checkout -b temp  
# Solve any conflicts between feature and master:
git rebase master 
# Reduce to a few commits:
git rebase -i master 
## An editor will open up, change the 'pick' to 'squash' on all 
## lines except the first one and other final commit.  

# Now update the commit message if you haven't already done that:
git commit --amend  
# Merge and clean up
git checkout master
git merge temp
git branch -D temp

Happy working with Git!