Our Gulp workflow for frontend projects and experiments

In this article, you will read about our workflow for frontend experiments. This workflow, based on Gulp, offers some advanced features like error reporting, live reloading, and source map generation.

<tl;dr> Full workflow on Github </tl;dr>

Introduction

Our workflow is constantly changing and being updated, this article reflects its state at the moment of writing (summer 2016). This article is by no means a tutorial or getting started guide, but serves as an insight into the way we work and the choices we have made. You can use our workflow as your own, as it is open-source code on Github, but will more likely serve as inspiration for creating or updating your own, as these things are highly opinionated.

This workflow allows us to easily initialize a project and test our it on multiple devices, with live-reloading. Our setup enables us to write code in languages of our own choice like SCSS instead of regular CSS. Furthermore, it allows us to use a file and folder structure of our own choosing but results in a minimum of (optimized) files. By loading less and smaller files, we minimize the load time of the page.

To start a project we just clone the repository from Git. After that we run npm install to get all the necessary packages defined in package.json. Finally we run gulp and we’re up and running.

File and folder structure

Most of our code for these small projects is written in the frontend folder. The files in here are compiled, minified and written to the public_html folder. The exception to this are the HTML files (in most cases this will be a single index.html) which still live in the public_html folder where they are edited.

The way we structure our SCSS folder is a subject for an article in the future. For now, you can check out the full file tree below.

/project
	/frontend
		/css
			screen.css
			screen.css.map
		/fonts
		/images
		/js
			/classes
			/libs
			/polyfills
			default.js
		/scss
			/elements
			/mixins
			/modules
			/utilities
			/variables
			/vendor
			screen.scss
	/node_modules
	/public_html
		/css
			screen.css
			screen.css.map
		/fonts
		/images
		/js
			default.js
			default.js.map
			libraries.js
			libraries.js.map
		index.html
	.gitignore
	gulpfile.js
	package.json
	README.MD

Live reloading

We utilize Browsersync for live reloading across multiple devices. Browsersync proxies our own virtual host and supplies an IP address which can be accessed by devices on the same network.

The task belows watches the generated files in the <code>public_html</code> folder for changes, to either reload the browser or stream the files without a full reload. The latter we use when editing the CSS of a page, so we maintain our scroll position on the page.

//Modules
var gulp = require('gulp'),
	browserSync = require('browser-sync'),
	reload = browserSync.reload;

//Config
var hostname = "dev.3dtimeline.nl";

//Watch task
gulp.task('watch',function(){
	browserSync({
		proxy: hostname
	},function(){
		//Watch CSS files to push to browser
		gulp.watch('public_html/**/*.css',function(){
			gulp.src('public_html/**/*.css').pipe(browserSync.stream());
		});
 		//Watch code files to reload browser
 		gulp.watch(['public_html/**/*.js','public_html/**/*.php','public_html/**/*.html'],function(){
			reload();
 		});
		//Watch image files to reload browser
		gulp.watch(['public_html/images/**/*.jpg','public_html/images/**/*.JPG','public_html/images/**/*.JPEG','public_html/images/**/*.png','public_html/images/**/*.gif'],function(){
			reload();
		});
	});
});

Error reporting

People who have experience using either a Gulp or Grunt based workflow might recognize the following scenario:

  1. You save your SCSS/LESS/CSS file and you wait for the task to finish and for your browser to reload.
  2. Nothing happens.
  3. You save again (repeatedly).
  4. Nothing happens.
  5. You reload manually (repeatedly).
  6. Nothing changes on screen.
  7. You repeat steps 3-6 until you look at your terminal and see you made a typo and your gulp/grunt task wouldn’t finish.
  8. You fix the error, and optionally restart your gulp watch task because the error was a fatal one and crashed your task.
  9. Continue.

To prevent this scenario we use a combination of both gulp-notify and gulp-plumber for error reporting and handling.

The gulp-notify package allows us to send desktop notifications when an error occurs. These notifications provide us with an audio and visual cue that something went wrong, without having to keep an eye on our terminal window.

Usually, when a gulp task runs into an error, the whole process will stop and you are required to restart the task again. The article here gives a brief explanation about this behavior. When something goes wrong, which eventually will happen, gulp-plumber prevents the whole task from shutting down.

In the example below, we define our onError function, which handles showing the notification. This function is called whenever gulp-plumber detects an error in the compass task.

var gulp = require('gulp'),
	compass = require('gulp-compass'),
	notify = require('gulp-notify'),
	plumber = require('gulp-plumber');

//Error notification
var onError = function(err,task) {
	var subtitle = err.plugin || err.message || err;
	console.log(err);
	notify({
		'title': 'Error executing '+task,
		'subtitle': message,
		'message': 'Check the Terminal for more information.',
		'sound':  'Hero',
		'icon': false,
		'contentImage': false,
		'open': 0,
		'wait': false,
		'sticky':true
	})(err);
};

//Task to compile scss files to css and run through autoprefixer
gulp.task('compass', function() {
	return gulp.src('frontend/scss/**/*.scss')
	.pipe(plumber({errorHandler: function(err){
		onError(err,'Compass')
		this.emit('end');
	}}))
	.pipe(compass({
		sass: 'frontend/scss/',
		css: 'frontend/css/',
		image: 'frontend/images',
		font: 'frontend/fonts',
		sourcemap:true
	}))
	.pipe(gulp.dest('./frontend/css'))
});

(S)CSS

We are really fond of SCSS as our choice of syntax for writing stylesheets. Like its cousin LESS it allows the use of variables, functions and nesting of rules. We’ve chosen for SCSS over LESS because the syntax just clicked with us. We used to rely heavily on the Compass framework for CSS3 mixins, but we use an autoprefixer for that now.

Below you can see the code with a task for compiling SCSS to CSS using Compass and running it through the autoprefixer. The result of this is a CSS file in frontend/css, which is not minified. This separate file sometimes comes in handy while debugging as it is easier to inspect than a minified file with sourcemaps. We have a second task for minifying of CSS and writing to the public_html/css folder.

var gulp = require('gulp'),
	compass = require('gulp-compass'),
	minifyCSS = require('gulp-cssnano'),
	sourcemaps = require("gulp-sourcemaps"),
	rename = require("gulp-rename"),
	autoprefixer = require("gulp-autoprefixer"),
	plumber = require('gulp-plumber');

//Variables for building
var supportedBrowsers = ['> 1% in NL'];

//Task to compile scss files to css and run through autoprefixer
gulp.task('compass', function() {
	return gulp.src('frontend/scss/**/*.scss')
 	.pipe(plumber({errorHandler: function(err){
		onError(err,'Compass')
		this.emit('end');
	 }}))
	 .pipe(compass({
		sass: 'frontend/scss/',
		css: 'frontend/css/',
		image: 'frontend/images',
		sourcemap:true
	}))
	.pipe(autoprefixer({
		browsers: supportedBrowsers,
	}))
	.pipe(gulp.dest('frontend/css'))
});

//Task to minify css files and write to theme folder
gulp.task('css',['compass'], function() {
	return gulp.src('frontend/css/**/*.css')
	.pipe(plumber({errorHandler: function(err){
		onError(err,'CSS')
		this.emit('end');
	}}))
	.pipe(sourcemaps.init({loadMaps: true}))
	.pipe(minifyCSS())
	.pipe(rename({suffix: '.min'}))
	.pipe(sourcemaps.write('.'))
	.pipe(gulp.dest('public_html/css'))
});

Image optimization

Every image saved in frontend/images is optimized and written to public_html/images. We like to keep access to the original files, which are easier to edit, which might happen in the case of SVGs.

We check with the gulp-newer npm package if a file in /frontend/images/ is newer than a file in /public_html/images, to prevent the optimization of all files when a single image is saved.

var gulp = require('gulp'),
	imagemin = require('gulp-imagemin'),
	plumber = require('gulp-plumber'),
	newer = require('gulp-newer');
	pngquant = require('imagemin-pngquant');

//Image optimisation
gulp.task('imageoptim', function () {
	return gulp.src(['frontend/images/**/*.svg','frontend/images/**/*.SVG','frontend/images/**/*.jpg','frontend/images/**/*.JPG','frontend/images/**/*.jpeg','frontend/images/**/*.JPEG','frontend/images/**/*.png','frontend/images/**/*.PNG','frontend/images/**/*.gif','frontend/images/**/*.GIF'])
	.pipe(plumber({errorHandler: function(err){
		onError(err,'Imageoptim')
		this.emit('end');
	}}))
	.pipe(newer('public_html/images'))
	.pipe(imagemin({
		progressive: true,
		svgoPlugins: [{removeViewBox: false}],
		use: [pngquant()]
	}))
	.pipe(gulp.dest('public_html/images'))
});

Javascript files

The organization of our Javascript files typically looks something like this.

/project
	/frontend
		/js
			/classes
				awesometimeline.js
			/libs
				jquery.js
			/polyfills
				requestanimationframe.js
			default.js
	/public_html
		/js
			default.js
			default.js.map
			libraries.js
			libraries.js.map
  • frontend/js/classes holds reusable classes.
  • frontend/js/polyfills holds polyfills for old browsers if needed
  • frontend/js/ generally holds a single default.js file, for initialization of classes and might get messy as it contains all code not contained in classes

We concatenate all these files (polyfills first, then our classes and then the default.js file) into a single minified file. Our task makes sure sourcemaps are included for easier debugging in the browser.

Javascript libraries

We concatenate all used Javascript libraries into a single file, which we minify using the gulp-uglify package. We don’t use an elaborate front-end package manager like Bower at the moment so we just place our library files in frontend/js/libs/ and use a single file to control the order in the concatenated file. This file, located at /frontend/js/libs/libs.js, uses the requires syntax. An example of this file can be seen here.

/**
 * Handle sort order of libraries
 *
 * @requires jquery.js
 * @requires jquery.mousewheel.js
 * @requires dragdealer.js
 */ 

The full task can be seen here:

var gulp = require('gulp'),
	plumber = require('gulp-plumber'),
	resolveDependencies = require('gulp-resolve-dependencies'),
	sourcemaps = require("gulp-sourcemaps"),
	concat = require('gulp-concat'),
	uglify = require('gulp-uglify'),
	rename = require("gulp-rename");

//JS library handling
gulp.task('js-libraries', function () {
	return gulp.src(['frontend/js/libs/libs.js'])
	.pipe(plumber({errorHandler: function(err){
		onError(err,'JS Libraries')
		this.emit('end');
	}}))
	.pipe(resolveDependencies({
		pattern: /\* @requires [\s-]*(.*\.js)/g
	}))
	.pipe(sourcemaps.init())
	.pipe(concat('libraries.js'))
	.pipe(uglify())
	.pipe(sourcemaps.write('./'))
	.pipe(gulp.dest('public_html/js'))
});

Watching gulpfile.js

A feature we don’t often see in other peoples workflow is watching the gulpfile.js itself for changes and restarting the gulp task when needed. This way you can easily test changes to your gulpfile without having to manually restart the gulp task in the terminal. This comes in especially handy when you are updating your gulpfile.

The default task below defines a function to spawn a child process in which our gulp watch task is called, on repeated calls it kills any existing process first. This way no manual restarts are necessary when updating the workflow.

var gulp = require('gulp'),
    child  = require('child_process');

gulp.task('default', function(){
    var ps;
    function spawn(){
        //Kill existing process if needed
        if(ps) ps.kill();
       
       //Start a new process
       ps = child.spawn('gulp', ['watch'], {stdio: 'inherit'})
    }

    //Call function to start our initial gulp process
    spawn();

    //On save of the gulpfile, call function to restart gulp process
    gulp.watch('gulpfile.js', function(event){
        spawn();
    });
});

Extras

We have some extra tasks in our gulpfile to make our life easier. These tasks are run manually.

  • Polyfills, a task to scan all our Javascript files (excluding libraries) and generate polyfills whenever they are needed and available for browsers we have defined.
  • Jshint, which utilizes the gulp-jshint package to check for errors and possible enhancements in our Javascript files.
  • Csshint, to check the syntax en style of our generated stylesheets using gulp-csshint.

Futur

Our workflow is constantly evolving. For instance, when working on our project Katwijk in Oorlog we have experimented with generating critical CSS and we might implement this in our core workflow. When any significant changes occur we will post a new article concerning the enhancements. We already have some topics we’re interested in looking at:

  • Implement a SCSS linter instead of a CSS linter.
  • Supply some decent documentation and usage guide.
  • Sprite generation.
  • Moving all the paths used in the gulpfile to a single config object on top of the file.
  • Load all development dependencies in our package.json automatically instead of all those require statements.
  • Better error-logging in the terminal, instead of logging the entire error object.

Closing comments

We hope you enjoyed this article! As this is one of the first articles we’ve published we’re really interested in your opinion, please leave your feedback and ideas in the comments section below!

<tl;dr> Full workflow on Github </tl;dr>

Comments