We have so far focussed on getting our application working and have not bothered much with setting things up right. This post is about setting up a good workflow for our frontend files and assets, that is, our html, css and javacript files, including the angular framework.
Using Spring for assets
As explained in this 2013 blog post, Spring will copy static resources that are located in
However, putting our javascript into this directory is getting tedious and we don't have any way to track versions and pull in updates to our javascript. On top of that, we would like to combine all our javascript in one file and minify that file. For these tasks, we need no fewer than three tools: Npm, Bower and Gulp.
src/main/resources/static for us at compile time. There are actually a few more directories where Spring will copy files from (such as src/main/resources/public) and it will put these files intarget/classes/static/ (or public, etc).However, putting our javascript into this directory is getting tedious and we don't have any way to track versions and pull in updates to our javascript. On top of that, we would like to combine all our javascript in one file and minify that file. For these tasks, we need no fewer than three tools: Npm, Bower and Gulp.
NPM
NPM apparently does not mean Node Package Manager, but it sure behaves a lot like that would be a good name for it. It requires
Basic characteristics of NPM are:
Using package.json:
node installed on your dev machine, which is no big deal.Basic characteristics of NPM are:
- NPM is based on its own npm repo, which has 1000s of packages in it. You can publish your own js pacakages to that.
- NPM is the first tool on the scene for most pipelines, it is used to pull other tools in via a
npm install boweretc.
- Basic command
npm install .... This will get you the most recent version and put it innode_modules/ - Variant for global installation (ie not in current directory, but in some
/usr/libfolder:npm install -g ... - Command to search:
npm search ..., you can also look at https://www.npmjs.com/ - More structured approach is to have a
package.jsonfile, see below for details. If that file is present, you can just runnpm installto download and set up all dependencies the project needs.
npm install; bower update to get all requirements in.Using package.json:
- First create a file
package.jsonwith{}as the contents - Then run
npm install ... --savefrom the command line and each install will update thepackage.jsonfor you! - Normally, the dependency will go into the
dependenciessection, if you runnpm install .. --save-devthe name will go into thedevDependenciessection of the package.json. This is what we do, see below. - You may want to edit your
package.jsonto specify exact version numbers, ie frombower: ^1.7.7(which means that version and up) remove the caret.
$ echo "{}" > package.json
$ npm install bower --save
$ npm install jshint --save
$ npm install gulp --save
Bower
Bower will record the files you install in
Setup of Bower for this project:
bower.json, the bower init will create a first version of that file for you.Setup of Bower for this project:
$ bower initJust answer all questions, leave 'main', 'moduletype' empty for now and accept defaults for 'ignore' and 'private'.
$ bower install angular#1.4.9 --save $ bower install angular-animate#1.4.9 --save $ bower install angular-uuid --save # told bower to choose the 1.4.9 version and record it $ bower install angular-cookies#1.4.9 --save
Useful bower commands:
- Install a new package with
bower install angular --save, similar to npm, this will download the package intobower_componentsand update thebower.jsonfile for you. - In my case, this downloaded Angular 1.5, while I wanted to stay on 1.4, so…
- To uninstall a package, run
bower uninstall angular --save, here the save flag removes the reference from the bower.json file. - You can use hash syntax to request a specific version,
bower install angular#1.4.9 --save. - If you forget the
--saveflag you can simply run the command again with the flag. - To see which packages you have installed, and which one can be updated,
bower list - To update a package, run
bower update ..
Dependencies in Bower
Bower keeps track of dependencies for you and will also record version numbers in the bower.json file, as we saw above. However, bower is not smart about dependencies, you will have to do that for it.
For example, I installed angular version 1.4.9. When I tried to install angular-animate however, the latest version of that package is 1.5.3 and it requires angular 1.5.3. So I will end up with a mixed situation, which is undesirable (to be fair, Bower does warn you about this). Bower does not work like
For example, I installed angular version 1.4.9. When I tried to install angular-animate however, the latest version of that package is 1.5.3 and it requires angular 1.5.3. So I will end up with a mixed situation, which is undesirable (to be fair, Bower does warn you about this). Bower does not work like
apt-get(the Debian/Ubuntu package installer) and others, which will ask you to upgrade angular or fail the installation of angular-animate.NPM vs Bower
Separation of concerns between NPM and Bower
- use
package.jsonandnpm installto keep your tools up to date - use
bower.jsonandbower ..to keep your front end dependencies up to date - This keeps your tools in
node_modulesand your dependencies inbower_components, which is convenient for further processing of the dependencies.
Using Gulp
Once, at the dawn of computer ages, there was
We have earlier talked about maven and gradle, the two main task runners in java land. For various reasons, javascript has its own task runners, with
We will first need a fair few gulp components. I added them with the code below, but you can just do
Finally, I created a symbolic link from my top level directory to the gulp binary, as follows
Gulp's settings and tasks are in
Gulp has a streaming concept, similar to functional programming or flow-based programming. Basically, you start a stream of files with a
make, which 'made' your computer program for your from sources. Make was the first of many task runners, programs that help you to compile sources, compress javascript, move assets around and package things up.We have earlier talked about maven and gradle, the two main task runners in java land. For various reasons, javascript has its own task runners, with
grunt and gulp as the two main contenders at the time of writing. We chose gulp here because it is generally easier to set up and it provides the watch functionality out of the box.We will first need a fair few gulp components. I added them with the code below, but you can just do
npm install.npm install main-bower-files --save npm install gulp-concat --save npm install gulp-uglify --save npm install gulp-print --save npm install gulp-rename --save npm install gulp-add-src --save
ln -s node_modules/gulp/bin/gulp.js gulp git add gulpThis works in all unixes (Linux and OS-X).
Gulp's settings and tasks are in
gulpfile.js, which is pretty human readable once you get used to the format: First, all dependencies are required and the results are assigned to variables. These variables are used as functions further down the line.Gulp has a streaming concept, similar to functional programming or flow-based programming. Basically, you start a stream of files with a
gulp.src command, then operate on that list of files in various steps. Each step is wrapped in a .pipe() function, similar to the use of pipes in bash.Moving the resources
Until now, our frontend resources lived in
We will have to move the resources now, Gulp can take care of copying things and if we leave them, they will be copied twice. I have chosen
Without explaining Gulp in full yet, copying files is easy: Below is the source for a
Then we do several operation on these files/filenames, each wrapped in a
We use two small magic tricks: In the
Read this excellent, pretty and short post on the principles of Gulp. This introduction helped me a ton.
src/main/resources/static and as we saw above, Spring will copy them to the target directory for us (Spring will copy everything under src/main/resources).We will have to move the resources now, Gulp can take care of copying things and if we leave them, they will be copied twice. I have chosen
src/main/frontend/ in this project. After you create the directory, make sure to right click it in IntelliJ and choose Mark Directory As ..: Sources Root. The folder will turn blue (in my color scheme) and IntelliJ wil index the files in this directory.Without explaining Gulp in full yet, copying files is easy: Below is the source for a
css task, which will copy all .css files from source to dest. It starts by declaring the task. Then we have a gulp.src call which will typically produce a list of files/filenames (Gulp bundles a filename and the file contents together in something it calls a Vinyl. Also note that technically, this is not a list but a stream).Then we do several operation on these files/filenames, each wrapped in a
.pipe() call as mentioned above. Our first step is print, which prints out the filename to the terminal. Our second and last step is gulp.dest, which will write the file contents out to the directory specified.We use two small magic tricks: In the
gulp.src, we use a javascript variable src instead of a full path; and we use /**/*.js to denote any js files in src or subdirectories of src (recursive copy).var print = require('gulp-print');
var src = 'src/main/frontend/';
var destprod = 'target/classes/static/';
// copy css files
gulp.task('css', function() {
gulp.src(src + '/**/*.css')
.pipe(print())
.pipe(gulp.dest(destprod));
});
Main-bower-files
Above, we introduced a separation between tools (managed with NPM) and front end libraries (managed with Bower). Now, we would like to combine and minify all this JS code into one file. There are standard Gulp plugins for these actions, but how do we get the names of all the frontend libraries we need?
The
You may have noted this version does not do any minification etc, and is called
The benefit of this approach is that production is fast, using minified files, while we can easily inspect and debug the full files in our development environment.
The
main-bower-files plugin can help us with that: It will essentially do a bower list command for us and create a list of front-end libraries to ship to the user, in the right order (if file B depends on file A, it should come after A in the combined js). Of course, we also have our own js file(s) and these should come after the library files. gulp-add-src plugin can take care of that.var bowerfiles = require('main-bower-files');
var addsrc = require('gulp-add-src');
var destdev = 'target/classes/static/devresources/'
// copy all js files for dev targets
gulp.task('js-dev', function() {
gulp.src(bowerfiles())
.pipe(addsrc.append(src + '**/*.js'))
.pipe(print())
.pipe(gulp.dest(destdev + 'js/'));
});
js-dev. It also copies to destdev instead of destprod. We have a matching js-prod, which concatenates all files together under the name 'steamvoat.js', then renames that to 'steamvote.min.js' (we could have combined and renamed in one step), then runs the 'uglify' minifier, and finally copies the files out to the production target directory. To avoid confusion, the prod tasks starts out by removing all the full-length dev files under the destdev directory.var concat = require('gulp-concat');
var uglify = require('gulp-uglify');
var rename = require('gulp-rename');
var del = require('del');
// minify and package all js files for production targets
gulp.task('js-prod', function() {
del(destdev);
gulp.src(bowerfiles())
.pipe(addsrc.append(src + '**/*.js'))
.pipe(print())
.pipe(concat('steamvoat.js'))
.pipe(rename({suffix: '.min'}))
.pipe(uglify())
.pipe(gulp.dest(destprod));
});
ProcessHtml
There is one small detail to deal with: We have replaced five js files with one combined and minified file in production, but the
You can see it in action at the tail end of our index.html. It starts with a html comment with
When run through ProcessHtml, this will show up as
The build commands should have modifier to make them work only on certain Grunt targets, but this does not seem to work for Gulp. This may be a good thing, now all the logic resides in the gulpfile.
In our gulpfile, there are therefore two rules for copying html files: A dev version which plain copies, and a production version which runs processHtml on the html.
<script src commmands on our main page do not reflect that. We could simply source all full files as browsers throw a harmless error when files are not found, but there is a better way. The ProcessHtml node module, which allows you to replace html, use templates and generally modify html to your needs.You can see it in action at the tail end of our index.html. It starts with a html comment with
build: as the first word. The 'build:js= subcommand will replace many lines of js scripts into one, the second argument specifies the name of the combined js file.<!-- build:js steamvoat.min.js --> <script src="/devresources/js/angular.js"></script> <script src="/devresources/js/angular-cookies.js"></script> <script src="/devresources/js/angular-uuid4.js"></script> <script src="/devresources/js/angular-animate.js"></script> <script src="/devresources/js/index.js"></script> <!-- /build --> </body> </html>
<script src="steamvoat.min.js"></script> </body></html>A good overview of what you can do with ProcessHtml is given in this blog post. For full information, you have to know that there is an underlying node version of ProcessHtml which does all the work, with specific grunt and gulp versions built on top of that. The best docs are over at the grunt version.
The build commands should have modifier to make them work only on certain Grunt targets, but this does not seem to work for Gulp. This may be a good thing, now all the logic resides in the gulpfile.
In our gulpfile, there are therefore two rules for copying html files: A dev version which plain copies, and a production version which runs processHtml on the html.
gulp watch
Half the reason for using Gulp is the excellent support for 'watching' files. Imagine that every time we change a html file, it automatically is copied over to the target directory so that our test server will use it if we refresh the page. This can be done by running
Now if you run
gulp watch// Watch for changes in files
gulp.task('watch', ['dev'], function() {
// Watch .js files
gulp.watch(src + '**/*.js', ['js-dev']);
// Watch html files
gulp.watch(src + '/**/*.html', ['html-dev']);
// Watch css files
gulp.watch(src + '/**/*.css', ['css']);
// Watch image files
gulp.watch(src + '/img/*', ['images']);
});
./gulp watch, gulp will build the dev version and then your terminal will seem to hang but whenever you make a change to any of your resources, it will run the appropriate target to copy the modified files over for you.Back to maven
To tie it all together, it would be great to have a unified interface to npm, bower and gulp. Also, we have various maven commands that we already need to build java, create a package for CloudFoundry etc. These take care of the java side, but our app will not work if the Gulp dev task has not run.
So how do we tie Gulp to Maven, our backend build tool? There is a great plugin "Front End Maven Plugin" for maven that takes care of all these tasks for us. I did not use it here because I did not want my gulp tasks to be run all the time and I don't want to install a local copy of node either. For larger projects with Continuous Integration, the plugin would be great. For this small size project, setting it up by hand was easy enough.
I use a small plugin to Maven that allows you to run any shell command. This does make the setup dependent on gulp and the exact location of gulp, but I can live with that.
First, we declare a plugin
So how do we tie Gulp to Maven, our backend build tool? There is a great plugin "Front End Maven Plugin" for maven that takes care of all these tasks for us. I did not use it here because I did not want my gulp tasks to be run all the time and I don't want to install a local copy of node either. For larger projects with Continuous Integration, the plugin would be great. For this small size project, setting it up by hand was easy enough.
I use a small plugin to Maven that allows you to run any shell command. This does make the setup dependent on gulp and the exact location of gulp, but I can live with that.
First, we declare a plugin
exec-maven-plugin. This plugin is run in the process-resources phase and it will run the exec goal. The plugin will run the executable mentioned in the configuration section, here gulp production:<build>
<plugins>
// ... stuff deleted ...
<plugin>
<artifactId>exec-maven-plugin</artifactId>
<groupId>org.codehaus.mojo</groupId>
<executions>
<execution><!-- Run our version calculation script -->
<id>Run Gulp</id>
<phase>process-resources</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>node_modules/gulp/bin/gulp.js</executable>
<arguments><argument>production</argument></arguments>
</configuration>
</plugin>
</plugins>
</build>
Making this work in IntelliJ
We want the gulp task 'production' to be run when we start the application from IntelliJ. There are two ways to do this.
The simplest way is to not use the IntelliJ configuration target (our main application file), but use the maven task
The nicer way is to set up an IntelliJ configuration to run the gulp task (see also IntelliJ help) and then make our main IntelliJ configuration run this gulp configuration by default. Here is how that goes:
Finally, we need to tell our main configuration (which runs our app) to run the gulp task first.
The simplest way is to not use the IntelliJ configuration target (our main application file), but use the maven task
spring-boot:run as the configuration to run. This has several drawbacks, mainly that you cannot configure how you run the target so well.The nicer way is to set up an IntelliJ configuration to run the gulp task (see also IntelliJ help) and then make our main IntelliJ configuration run this gulp configuration by default. Here is how that goes:
- Go to the configurations dropdown of your IntelliJ. For me, this sits in the top right corner. Choose 'Edit Configurations'. Alternatively, go to
Run: Edit Configurations. - Create a new configuration with the plus button (top left)
- From the dropdown, choose Gulp.js
- An 'unnamed' gulp task is created, call it 'production' (or whatever)
- Make sure that the gulpfile textbox points to your gulpfile.js
- Under task, select 'production'
- All other settings should be fine and can be left open if they are blank.
Run: Run.... A 'gulp' tab should open on your "Run" view (bottom of IntelliJ screen) and you should see the production task copy files and conclude that with a satisfying "Process finished with exit code 0".Finally, we need to tell our main configuration (which runs our app) to run the gulp task first.
- Go to Edit Configurations again
- Under 'Spring Boot', you should see a configuration that runs your app. Click on the configuration.
- On the bottom of the window, there is a box called "Before Launch: .." with one task in it, called "Make" (if you do not see that box, stop all running processes and run this configuration once).
- Click the plus button, then click "Run another configuration" and select your gulp 'Production' task.
- Use the triangle icons to have Make be the last item.
Final workflow
When a new clone of the project is made, you will have to run a few commands to get the dependencies:
To build and run the project, you can use maven or IntelliJ, as outlined above.
When making changes to the frontend files, you can view the changes with a simple browser reload if you run a watch before you start editing:
npm install bower updateYou will also have to run these commands every now and then to pick up new versions of the packages you are using. I think this is a great workflow, as it means you will not be surprised by new versions. You have to remember to run the commands on occasion.
To build and run the project, you can use maven or IntelliJ, as outlined above.
When making changes to the frontend files, you can view the changes with a simple browser reload if you run a watch before you start editing:
gulp watchAnd that is it!
Conclusion
This seems a lot of work but it is definitely worth it if you want to maintain your app easily and if you want to automate the build and deploy process.
