Workflow Automation for web applications – NPM – Bower



Workflow Automation for web applications – NPM – Bower

1 1


workflow-automation

Front-end workflow automation

On Github maxyermayank / workflow-automation

Workflow Automation for web applications

Created by Mayank Patel / @maxy_ermayank

Why workflow automation?

  • Boilerplate
  • Dependency management
  • Framework
  • Abstrations
  • Build
  • Automation test
  • Docs
  • Continuous integration
  • Deployment
  • Performance optimization
  • Workflow
  • Deployment

Work Flow

Setup

  • Download dependecies
  • Download frameworks
  • Download libraries
  • Scaffolding

Develop

  • Non minified
  • Linting (HTML, JS)
  • Seperated files
  • Generate responsive images
  • Optimize images
  • Compilation (CoffeScript, SASS, LESS..)
  • Test configuration
  • Unit testing & e2e testing
  • Generate test report
  • Watchers
  • Live reload

Build

  • Annotate (JS)
  • Generate copyright and license information
  • Sourcemap (JS, CSS)
  • Concatenation (JS, CSS)
  • Minification (HTML, JS, CSS)
  • Uglification (HTML, JS, CSS)
  • Compress (JS, CSS)
  • Live configuration
  • Compiled
  • Renamed
  • Cache templates (HTML)
  • Inject resources in Template
  • Optimize performance
  • Deployment setup

Dependency Management Tools

Downloads dependencies using Git, HTTPS, ZIP, npm

  • npm
  • Bower

NPM

  • Package manager for the web
  • Comes with node.js
  • Default for node.js modules

Install node.js

Filename: package.json

Download node.js and follow installation guide

npm examples

Find available package Here.

  > npm install <package>

  -g installs package globally
  > npm install -g <package>

  use --save to save dependecy in package.json & -dev to lock package version 
  > npm install <package> --save-dev
  > npm init

Sample Package.json

{
    "name": "project-name",
    "version": "1.0.0",
    "description": "Description goes here",
    "main": "index.html",
    "scripts": {
        "test": "gulp e2e"
    },
    "repository": {
        "type": "git",
        "url": "https://example.com/project-name"
    },
    "author": "Mayank Patel <maxy.ermayank@gmail.com>",
    "license": "MIT",
    "bugs": {
        "url": "https://example.com/project-name/issues"
    },
    "homepage": "https://example.com/project-name",
    "dependencies": {},
    "devDependencies": {
        "angular": "^1.3.15",
        "gulp": "^3.8.5",
        "gulp-connect": "^2.0.5",
        "gulp-shell": "^0.2.9",
        "karma": "^0.12.21",
        "karma-chrome-launcher": "^0.1.4",
        "karma-jasmine": "^0.1.5",
        "karma-requirejs": "^0.2.2",
        "lodash": "^2.4.1",
        "protractor": "^1.0.0",
        "requirejs": "^2.1.14"
    },
    "engines": {
        "node": ">=0.8.0"
    },
    "keywords": [
        "gruntplugin"
    ],
}

Demo time

Bower

  • Package manager for the web
  • Designed solely for web and it is optimized with that in mind.

Install

Filename: bower.json

    Install Globally
    > npm install -g bower

    Install in your project
    > npm install bower

Examples

Packages available Here

    bower install <package>

    bower install git://github.com/user/package.git

    bower install http://example.com/script.js

Sample bower.json

{
  "name": "project-name",
  "version": "1.0.0",
  "author": "Mayank Patel <maxy.ermayank@gmail.com>",
  "homepage": "https://example.com/project-name",
  "description": "Description goes here",
  "main": "index.html",
  "license": "apache 2",
  "dependencies": {
    "angular": "1.3.15",
    "json3": "~3.2.4",
    "es5-shim": "~2.0.8",
    "angular-resource": "1.3.15",
    "d3": "3.3.x"
  },
  "devDependencies": {
    "angular-mocks": "1.2.0-rc.1",
    "angular-scenario": "1.2.0-rc.1"
  }
}

Demo time

Scaffolding Tool / Generator

Yemon (YO)

  • Scaffolds out boilerplate
  • Abstraction
  • Performance optimization
  • Testing and Build process
  • Custom generators are available

    Install YO globally
    > npm install -g yo

examples

> yo
[?] What would you like to do? 
›❯ Install a generator
Run the Angular generator (0.4.0) 
Run the Backbone generator (0.1.9) 
Run the Blog generator (0.0.0)
Run the jQuery generator (0.1.4) 
Run the Gruntfile generator (0.0.6)
(Move up and down to reveal more choices)
yo jquery-boilerplate
Boom. You just created a jQuery plugin.

Custom generator

Find available generators Here.

> npm install generator-bootstrap -g
> yo bootstrap

> npm install generator-webapp -g
> yo webapp

Generate enterprise app using Angular

> npm install generator-angular -g
> yo angular
> yo angular:view user
> yo angular:controller user
> yo angular:directive mydirective

Demo time

Build Tool

Gulp Strength

  • Mature: ~ Aug 2013, relatively mature
  • Community Support: New kid in town, Picking up popularity
  • Code over configuration
  • Easy to read & use
  • Tons of plugins available
  • Provides node streams - no need for tmp files/folders
  • Plugins do ONE thing
  • Provides plugin to run Grunt tasks
  • Only has 5 functions to learn!
  • Runs with maximum concurrency by default

Gulp Weeknesses

  • Can be daunting to learn streams
  • Sometimes setting up src/dest can be tricky (use base)
  • Streams are not a great abstraction for complex builds. They don't compose well.

Install Gulp

Filename: gulpfile.js

Install Gulp in your project
> npm install gulp --save
Install Gulp Globaly
> npm install -g gulp

The gulp api

  • gulp.task(name[, deps], fn)
  • gulp.src(globs)
  • gulp.dest(path)
  • gulp.watch(glob[, opts], tasks)
  • gulp.run(tasks...)

gulp.task(name[, deps], fn)

gulp.task('somename', function() {
// Do stuff
});
gulp.task('build', ['somename', 'test'];
> gulp build

gulp.src(globs)

gulp.src('client/templates/*.jade')
      .pipe(jade())
gulp.src(['src/**/*.js', 'test/spec/**/*.js'])
    .pipe(jshint())

gulp.dest(path)

gulp.src('./client/templates/*.jade')
      .pipe(jade())
      .pipe(gulp.dest('./build/templates'))
      .pipe(minify())
      .pipe(gulp.dest('./build/minified_templates'));

gulp.watch(glob[, opts], tasks)

gulp.watch('app/**/*.js', ['test','reload']);

gulp.run(tasks...)

gulp.task('hello-world', function () {
  run('echo Hello World').exec()  // prints "[echo] Hello World\n".
    .pipe(gulp.dest('build'))    // Writes "Hello World\n" to output/echo.
})

Streams

gulp.task('scripts', function () {
      return gulp.src('src/app/**/*.js') // <-- read from filesystem
        // In memory transform
        .pipe(jshint('.jshintrc'))    // <-- lint the code
        .pipe(concat('app.min.js'))   // <-- concatenate to one file
        .pipe(uglify())               // <-- minify the file
        .pipe(rev())                  // <-- add revision to filename
        .pipe(gulp.dest('dist/app')); // <-- write to filesystem
    });

Sample gulpfile.js

'use strict';

var gulp = require('gulp'),
gutil = require('gulp-util'),
del = require('del'),
jshint = require('gulp-jshint'),
ngAnnotate = require('gulp-ng-annotate'),
concat = require('gulp-concat'),
sourcemaps = require('gulp-sourcemaps'),
uglify = require('gulp-uglify'),
concatCss = require('gulp-concat-css'),
minifyCSS = require('gulp-minify-css'),
imagemin = require('gulp-imagemin'),
minifyHtml = require('gulp-minify-html'),
templateCache = require('gulp-angular-templatecache'),
inject = require('gulp-inject'),
arialinter = require('gulp-arialinter'),
jasmine = require('gulp-jasmine'),
protractor = require("gulp-protractor").protractor,
webdriver_standalone = require("gulp-protractor").webdriver_standalone,
header = require('gulp-header'),
connect = require('gulp-connect'),
open = require('gulp-open');
var port = 8001;
//serve = require('gulp-serve');

var paths = {

    dist: 'dist',

    jsSource: ['src/scripts/*.js'],

    styleSource: ['src/css/main.css'],

    imagesSource: ['src/images/**/*.*'],

    htmlSource: ['src/views/**/*.html'],
    unitTestSource: ['test/unit/**/*.js'],
    e2eSource: ['test/e2e/*.js'],

    vendorScripts: [
             'bower_components/jquery/dist/jquery.min.js',
             'bower_components/bootstrap/dist/js/bootstrap.min.js',
             'bower_components/underscore/underscore-min.js',
             'bower_components/angular/angular.min.js',
             'bower_components/angular-resource/angular-resource.min.js',
             'bower_components/angular-animate/angular-animate.min.js',
             'bower_components/angular-messages/angular-messages.min.js',
             'bower_components/angular-sanitize/angular-sanitize.min.js',
             'bower_components/angular-aria/angular-aria.min.js'
    ],

    vendorStyles: [
                'bower_components/bootstrap/dist/css/bootstrap.min.css',
                'bower_components/bootstrap/font-awesome/css/font-awesome.min.css'
       ]
};

gulp.task('clean', function (cb) {
    del(paths.dist, cb);
});

gulp.task('vendorScripts', function() {

    return gulp.src(paths.vendorScripts)
            .pipe(concat('scripts/vendor/vendor.js.gzip'))
            .pipe(uglify({compress : true}))
            .pipe(gulp.dest(paths.dist));
});

gulp.task('vendorStyles', function() {

    return gulp.src(paths.vendorStyles)
            .pipe(concatCss('css/vendor/vendor.css'))
            .pipe(gulp.dest(paths.dist));
});

gulp.task('lint', function() {
    return gulp.src(paths.jsSource)
            .pipe(jshint())
            .pipe(jshint.reporter('default'));
});

gulp.task('buildScript', function() {

    return gulp.src(paths.jsSource)
            .pipe(ngAnnotate({remove: true, add: true, single_quotes: true}))
            .pipe(concat('scripts/app.js'))
            .pipe(header('/** Copyright '+new Date().getFullYear()+' Author: Mayank Patel **/ \n'))
            .pipe(gulp.dest(paths.dist));
});

gulp.task('minifyScript', ['buildScript'], function() {

    return gulp.src('dist/scripts/app.js')
            .pipe(concat('scripts/app.min.js'))
            .pipe(sourcemaps.init())
            .pipe(uglify())
            .pipe(sourcemaps.write('/'))
            .pipe(gulp.dest('dist'));
});

gulp.task('compressScript', ['minifyScript'], function() {

    return gulp.src('dist/scripts/app.min.js')
            .pipe(concat('scripts/app.min.js.gzip'))
            .pipe(uglify({compress : true}))
            .pipe(gulp.dest(paths.dist));
});

gulp.task('scripts', ['lint', 'compressScript'], function() {

});


gulp.task('buildStyles', function() {
    return gulp.src(paths.styleSource)
            .pipe(concat('css/main.css'))
            .pipe(gulp.dest(paths.dist));
});


gulp.task('minifyStyles', ['buildStyles'], function() {

    return gulp.src('dist/css/main.css')
            .pipe(concat('css/main.min.css'))
            .pipe(sourcemaps.init())
            .pipe(minifyCSS())
            .pipe(sourcemaps.write('/'))
            .pipe(gulp.dest(paths.dist));
});

gulp.task('compressStyles', ['minifyStyles'], function() {

    return gulp.src('dist/css/main.min.css')
            .pipe(concat('css/main.min.css.gzip'))
            .pipe(uglify({compress : true}))
            .pipe(gulp.dest(paths.dist));
});

gulp.task('styles', ['minifyStyles'], function() {
});

gulp.task('images', function() {
    return gulp.src(paths.imagesSource)
        .pipe(imagemin({progressive: true, optimizationLevel: 5, interlaced: true}))
        .pipe(gulp.dest('dist/images'));
});

gulp.task('views', function() {

    gulp.src(paths.htmlSource)
        .pipe(arialinter({ level: 'AA' }))
        .pipe(templateCache({ module: 'blog', root: 'src/views' }))
        .pipe(gulp.dest('dist/scripts'));

    gulp.src('src/index.html')
        .pipe(arialinter({ level: 'AA' }))
        //.pipe(minifyHtml({ conditionals: true, spare:true }))
        .pipe(gulp.dest('dist'));

    gulp.src(['src/views/**/*']).pipe(minifyHtml()).pipe(gulp.dest('dist/views'));
});

gulp.task('build', ['vendorScripts', 'vendorStyles', 'scripts', 'styles', 'images', 'views'], function() {
});

gulp.task('test', function() {

    //gulp.src(paths.unitTestSource)
    //.pipe(jasmine({reporter: new reporters.JUnitXmlReporter()}));
});

gulp.task('connect', function () {
  connect.server({
    root: 'dist',
    port: port,
    livereload: true,
    // https: true,
    // middleware: function(connect, opt) {
    //   return [
    //     // ... 
    //   ]
    // }
  });
});

gulp.task('open', function(){
  var options = {
    url: 'http://localhost:' + port,
    //app: 'Google Chrome'
    app: 'Google Chrome Canary'
    // app: 'Firefox'
    // app: 'Safary'
  };
  gulp.src('./dist/index.html')
  .pipe(open('', options));
});

gulp.task('serve', ['connect', 'open']);

gulp.task('e2eTest', function() {

    gulp.task('webdriver_standalone', webdriver_standalone)

    gulp.src(paths.e2eSource)
        .pipe(protractor({
            configFile: "test/protractor.conf.js",
            args: ['--baseUrl', 'http://localhost:8001/dist/index.html']
        })) 
        .on('error', function(e) { throw e })
});

gulp.task('watch', function() {
    gulp.watch(paths.jsSource, ['scripts']);
    gulp.watch(paths.imagesSource, ['images']);
    gulp.watch(paths.styleSource, ['styles']);
    gulp.watch(paths.htmlSource, ['views']);
});

gulp.task('default', ['clean', 'build', 'test', 'serve']);//, 'e2eTest'

Demo time

Grunt Strength

  • Mature: Mar 2012, very mature
  • Community Support: Most Popular
  • Configuration over code
  • Based on files
  • Tons of plugins available
  • Flexibility
  • Scaffolding is available through generators
  • Provides plugin to run Gulp tasks

Grunt Weeknesses

  • Plugins do multiple things
  • Headache of temp files/folders
  • Not one solid control flow
  • Configuration can get lengthy - 500+ lines / Hard to read
  • Very lengthy & vast API
  • Can get pretty slow when tasks increase

Install Grunt

Filename: gruntfile.js

Install Grunt in your project
> npm install grunt --save
Install Grunt-cli / Global install of Grunt command line
> npm install -g grunt-cli

Structure

module.exports = function(grunt) {

  grunt.initConfig({
    // Configuration des tâches
  });

  // Enregistrement d'une tâche
  grunt.registerTask(taskName, [description, ] taskFunction)

  // Chargement d'un plugin
  grunt.loadNpmTasks('package');

};

Demo time

Brunch

  • Mature: Jan 2011, very mature
  • Community Support: fairly new, plenty of plugins & skeletons
  • Easy to set up - use skeleton
  • Introduces conventions for you
  • Simple CLI - only a few commands
  • Commands for dev/staging/production releases

Brunch Weeknesses

  • Not using conventions causes headaches
  • Not easy to set up with existing projects
  • Skeleton set ups not maintained as fast as plugins
  • Not as supported as Grunt/Gulp

Install Brunch

Filename: brunch-config.js

> npm install -g brunch

Demo time

//Create new skeleton of angular app
brunch new https://github.com/scotch/angular-brunch-seed myapp

//Install bower packages (css/js/etc)
bower install

//tell Brunch to watch your project and incrementally rebuild it when source files are changed
brunch watch --server

//builds a project for distribution. By default it enables minification
brunch build --production

Broccoli

  • Mature: Feb 2014, still in beta
  • Community Support: in Beta
  • Trees allow dev to think of assets
  • Provides caching for map files
  • Makes some conventions for you - assets
  • Watching files handled by serve, only rebuilds whats needed
  • Supported by trees that mirror filesystem
  • Lets you have a transactional-style build pipeline
  • Trees later in process can have more files than those previously
  • Can do operations on multiple generated trees
  • Can merge trees (output for ES5 from TypeScript & ES6 sources for example)
  • Incremental builds by default
  • Cacheing is built-in rather than manual

Broccoli Weeknesses

  • No parallelism
  • Mainstream usage only in ember-cli (and even then only used as a lib, not as a tool itself)
  • Many rough edges - not forgiving for things off common path
  • Some implementation flaws (though fixable)
  • Some design decisions won’t scale from company to company, though it maps better to blaze (bazel) than does gulp

Install Broccoli

Filename: brocfile.js

Install Broccoli globaly
> npm install --g broccoli-cli
Install Broccoli in your project
> npm install --save-dev broccoli

Sample brocfile.js

var pickFiles = require('broccoli-static-compiler');
var mergeTrees = require('broccoli-merge-trees');
var EmberApp = require('ember-cli/lib/broccoli/ember-app');

var app = new EmberApp({
  name: require('./package.json').name,

  minifyCSS: {
    enabled: true,
    options: {}
  },

  getEnvJSON: require('./config/environment')
});

// Use this to add additional libraries to the generated output files.
app.import('vendor/ember-data/ember-data.js');

// If the library that you are including contains AMD or ES6 modules that
// you would like to import into your application please specify an
// object with the list of modules as keys along with the exports of each
// module as its value.
app.import('vendor/ic-ajax/dist/named-amd/main.js', {
  'ic-ajax': [
    'default',
    'defineFixture',
    'lookupFixture',
    'raw',
    'request',
  ]
});

var bootstrapTree = pickFiles('vendor/bootstrap/dist/css', {
  srcDir: '/',
  files: ['bootstrap.min.css'],
  destDir: '/assets'
});

module.exports = mergeTrees([app.toTree(), bootstrapTree]);

Demo time

Test Coverage

Test Frameworks

  • Jasmine
  • Karma
  • Istanbul
  • Protractor

Jasmine

  • Started in 2010
  • Huge community - Most popular
  • Behavior-driven development framework for testing JavaScript code
  • Doesn't require DOM, can be used serverside or in the browser
  • Obvious syntax
  • Easy to write tests
  • Async Support
  • Continuous Integration

Install Jasmine

Install jasmine globaly
> npm install -g jasmine

Install jasmine plugin for Grunt/Gulp
> npm i grunt-jasmine-runner --save-dev
> npm install gulp-jasmine --save-dev

Basics

describe("Test suite", function() {
  it("contains spec with an expectation", function() {
    expect(true).toBe(true);
  });
});

Jasmine Usage with Gulp

var gulp = require('gulp');
var jasmine = require('gulp-jasmine');

gulp.task('default', function () {
    return gulp.src('spec/test.js')
        .pipe(jasmine());
});
> gulp

Available Matchers

  • toBe()
  • toEqual()
  • toMatch()
  • toBeDefined()
  • toBeUndefined()
  • toBeNull()
  • toBeTruthy()
  • toBeFalsy()
  • toContain()
  • toBeLessThan()
  • toBeGreaterThan()
  • toBeCloseTo()
  • toThrow()

Above matchers can be chained with the Not() function. e.g. not.toBe()

Karma

  • Executes tests and source in a browser
  • Lots of plugins available
  • Can drive multiple browsers at once
  • Built in JUnit reporter

Install Karma

Install Karma command line tool globaly
> npm install -g karma-cli

Install Karma in project
> npm install karma --save-dev
> karma init

Karma Configuration

module.exports = function(config) {
  config.set({

    basePath: '',

    frameworks: ['jasmine','browserify'],

    files: ['test/spec/**/*Spec.coffee'],

    preprocessors: {
        'test/spec/**/*.coffee': ['coffee', 'browserify']
    },

    port: 9876,

    browsers: ['Chrome', 'Firefox', 'PhantomJS']

  });
};

Istanbul

  • Instrument your source code
  • Run your test suite against your instrumented source code
  • Store your coverage results
  • Allows you to generate coverage report
  • HTML and LCOV reporting

Install

> npm install istanbul

> npm install grunt-istanbul --save-dev

> npm install gulp-istanbul --save-dev

Protractor

  • AngularJS E2E Testing Framework
  • Built on Selenium's WebDriver API
  • Built on top of Jasmine framework
  • Extension for all browsers
  • Every action is asynchronous.
  • Rapid development.
  • Allows to test your app the way end user will use it.

Install Protractor

Install protractor globally
> npm install protractor -g

Install protractor in your project using Grunt
> npm i grunt-protractor-runner --save-dev

Install protractor in your project using Gulp
> npm install gulp-protractor --save-dev

Scaffolding is available through YO as well
> npm install -g generator-protractor
> yo protractor 

Update WebDriver
> webdriver-manager update

Sample Configuration File

exports.config = {
  seleniumAddress: 'http://localhost:4444/wd/hub',
  specs: ['spec.js'],
  multiCapabilities: [{
    browserName: 'firefox'
  }, {
    browserName: 'chrome'
  }]
}

Sample spec.js

describe('angularjs homepage', function() {
  var firstNumber = element(by.model('first'));
  var secondNumber = element(by.model('second'));
  var goButton = element(by.id('gobutton'));
  var latestResult = element(by.binding('latest'));
  var history = element.all(by.repeater('result in memory'));

  function add(a, b) {
    firstNumber.sendKeys(a);
    secondNumber.sendKeys(b);
    goButton.click();
  }

  beforeEach(function() {
    browser.get('http://juliemr.github.io/protractor-demo/');
  });

it('should have a history', function() {
    add(1, 2);
    add(3, 4);

    expect(history.last().getText()).toContain('1 + 2');
    expect(history.first().getText()).toContain('3 + 4');
  });
});

Execute e2e Test

> webdriver-manager start

> protractor <path to conf file>

Protractor Api

Selectors

  • by class name
  • by css
  • by id
  • by linkText
  • by partialLinktext
  • by name
  • by xpath
  • by binding
  • by input
  • by repeater
  • by model

Methods

  • clear()
  • click()
  • getAttribute(name)
  • getCSSValue(propertyName)
  • getLocation()
  • getSize()
  • getTagName()
  • getText()
  • isDisplayed()
  • isEnabled()
  • isSelected()
  • sendKeys(keysToSend)

Demo time

Choosing workflow automation tool

  • No tool is wrong, just different approaches
  • Each tool has strengths and weaknesses.
  • Your job will be to identify which tool may be best suited for your needs.

Developer tools

Let's make smart Applications

ARIA (Accessible Rich Internet Applications specification) W3C Standard WAVE (Web Accessibility Evaluation Tool) Plugin

Demo Time

Resources

Thank You @maxy_ermayank

Workflow Automation for web applications Created by Mayank Patel / @maxy_ermayank