Introducing Mongo-Up

A month or so ago, I was in need of a Node-based utility to deploy changes or data migrations of my Mongo DB database.  I work in an industry that has high compliance needs, so it wasn’t appropriate for us to do manual scripting of changes.  And logging onto our production Mongo was out of the question.

I started by using a project called Migrate-Mongo, but because we use AWS and I needed to retrieve configuration dynamically, I had to add some async configuration functionality and a pull request was accepted.

Next, I needed a way to run scripts that I wanted to happen idempotently with every release.  I added support for scripts that were run before and after every release (like ensuring indexes).  This PR added some significant complexity and was also a significant rewrite, but I added a ton of tests and made sure that coverage was at 100%.

Understandably, the Migrate-Mongo owner didn’t want to incorporate such a large change to the original project. But that’s the great thing about OSS, I just created Mongo-Up to make this great functionality to all of you great peeps!

Check it out, use it, improve it!

NPM: https://www.npmjs.com/package/mongo-up

GitHub: https://github.com/unbill/mongo-up

 

Nozus JS 1: Intro to Sails with Passport and JWT (JSON Web Token) Auth

This project extends from some previous posts on creating a SPA style application with Node (Sails) and Aurelia. I’m not going to go into detail about installing Node, NPM, or Sails, other than when it’s germane to the subject.  I’m assuming you are already set up with all of the needed basics for Node/Sails development. If not, visit the Sails site to get started.

We’ll start by creating a new directory in which to create our test projects and bringing up a command line. What we’re creating here is an API; so no need for any front end mixed into the Sails application.  We can create that later if we like. So the command to create a new app without a front end is:

sails new myApi --no-frontend

The basics of the app should now be present in the “myApi” directory (or whatever you called it). If you open up the project in your favorite editor, you will see the following structure:

structure

Most of the action will happen in the api folder but we’ll also need to set up some config settings. But first let’s go ahead and install some dependencies. Make sure you are in the root folder of your project and install the following from the command line:

npm install jsonwebtoken --save
npm install bcrypt-nodejs --save
npm install passport --save
npm install passport-jwt --save
npm install passport-local --save

Alternately, you could just add these to the package.json file and run npm install.

Config

Now we’ll add our passport configuration.  Another nice thing about Sails is that putting a js file in the /config folder means that it will be run when you “lift” or start the app. In the config folder, create passport.js and add the following code:


/**
 * Passport configuration file where you should configure strategies
 */
var passport = require('passport');
var LocalStrategy = require('passport-local').Strategy;
var JwtStrategy = require('passport-jwt').Strategy;

var EXPIRES_IN_MINUTES = 60 * 24;
var SECRET = process.env.tokenSecret || "4ukI0uIVnB3iI1yxj646fVXSE3ZVk4doZgz6fTbNg7jO41EAtl20J5F7Trtwe7OM";
var ALGORITHM = "HS256";
var ISSUER = "nozus.com";
var AUDIENCE = "nozus.com";

/**
 * Configuration object for local strategy
 */
var LOCAL_STRATEGY_CONFIG = {
  usernameField: 'email',
  passwordField: 'password',
  passReqToCallback: false
};

/**
 * Configuration object for JWT strategy
 */
var JWT_STRATEGY_CONFIG = {
  secretOrKey: SECRET,
  issuer : ISSUER,
  audience: AUDIENCE,
  passReqToCallback: false
};

/**
 * Triggers when user authenticates via local strategy
 */
function _onLocalStrategyAuth(email, password, next) {
  User.findOne({email: email})
    .exec(function (error, user) {
      if (error) return next(error, false, {});

      if (!user) return next(null, false, {
        code: 'E_USER_NOT_FOUND',
        message: email + ' is not found'
      });

      // TODO: replace with new cipher service type
      if (!CipherService.comparePassword(password, user))
        return next(null, false, {
          code: 'E_WRONG_PASSWORD',
          message: 'Password is wrong'
        });

      return next(null, user, {});
    });
}

/**
 * Triggers when user authenticates via JWT strategy
 */
function _onJwtStrategyAuth(payload, next) {
  var user = payload.user;
  return next(null, user, {});
}

passport.use(
  new LocalStrategy(LOCAL_STRATEGY_CONFIG, _onLocalStrategyAuth));
passport.use(
  new JwtStrategy(JWT_STRATEGY_CONFIG, _onJwtStrategyAuth));

module.exports.jwtSettings = {
  expiresInMinutes: EXPIRES_IN_MINUTES,
  secret: SECRET,
  algorithm : ALGORITHM,
  issuer : ISSUER,
  audience : AUDIENCE
};

So there’s a few things going on here.  First, we’re importing both passport and the two strategies we’re going to set up for now (local and JWT). We’ll add social login in the next post. Next, we’re going to add configuration for our two auth strategies. For the local strategy we’ll use email and password in to login. For JWT, we’ll set up the parameters needed to verify the token: secret, issuer and audience.  It should be noted again that the issuer and audience are optional but provide some additional verification.

Next we’ll add some functions to react to the auth request for each strategy. For local auth, this means making sure that the user matches the user in our database and returning a false response with a message on failure or the user if it succeeded.  For JWT auth, the strategy is already validating the token internally. If you want additional validation here you can check the user ID against the database to verify that this user actually exists and is active.  For the sake of simplicity, I’m going to trust the token and retrieve the user information from the token payload and just return it.

Last, we are exporting some settings to be used elsewhere in the API. This is another nice feature of Sails: The ability to export settings in a config file to make them available globally. So in our case, we are exporting jwtSettings from our config. To access these settings from anywhere, we can use the global accessor:  sails.config.jwtSettings.

Services

Next add a new file under api/services called CipherService. In sails, services and models are named using PascalCase by convention. I need a better name for this one but it will do for the time being. This is where we’ll put all of our code that does hashing and/or token stuff.

var bcrypt = require('bcrypt-nodejs');
var jwt = require('jsonwebtoken');

module.exports = {
  secret: sails.config.jwtSettings.secret,
  issuer: sails.config.jwtSettings.issuer,
  audience: sails.config.jwtSettings.audience,

  /**
   * Hash the password field of the passed user.
   */
  hashPassword: function (user) {
    if (user.password) {
      user.password = bcrypt.hashSync(user.password);
    }
  },

  /**
   * Compare user password hash with unhashed password
   * @returns boolean indicating a match
   */
  comparePassword: function(password, user){
    return bcrypt.compareSync(password, user.password);
  },

  /**
   * Create a token based on the passed user
   * @param user
   */
  createToken: function(user)
  {
    return jwt.sign({
        user: user.toJSON()
      },
      sails.config.jwtSettings.secret,
      {
        algorithm: sails.config.jwtSettings.algorithm,
        expiresInMinutes: sails.config.jwtSettings.expiresInMinutes,
        issuer: sails.config.jwtSettings.issuer,
        audience: sails.config.jwtSettings.audience
      }
    );
  }
};

We’ll use these methods elsewhere. The hash/comparePassword methods are pretty self-explanatory. Bcrypt-nodejs even adds in a salt to the hashed password by default. Pretty nice. The createToken method uses JsonWebToken to create a new token using the sign method. This method accepts a payload (the user in our case), a secret to use for the self-contained JWT hash, and some metadata including the algorithm, expiration, issuer and audience. Issuer and Audience will also be validated when the token is verified coming back in, so it gives a little extra protection. The inclusion of the user in the payload is really just for example. In the real world, you might include the user id and claims etc…

API Generation

Next we’ll create our User API. Return to the root command line of the application and run the following command:

sails generate api User

If you now observe your controllers and models directories, you’ll see the generated controller and model for User.  The controller can be left as-is for now. Although it looks empty, it’s functional!  It will use default blueprints included with Sails to provide functionality. Now we’ll want to enhance the model.  Open the api/models/User file and add the following code:

/**
 * User
 * @description :: Model for storing users
 */
module.exports = {
    schema: true,
    attributes: {
        username: {
            type: 'string',
            required: true,
            unique: true,
            alphanumericdashed: true
        },
        password: {
            type: 'string'
        },
        email: {
            type: 'string',
            email: true,
            required: true,
            unique: true
        },
        firstName: {
            type: 'string',
            defaultsTo: ''
        },
        lastName: {
            type: 'string',
            defaultsTo: ''
        },
        photo: {
            type: 'string',
            defaultsTo: '',
            url: true
        },
        socialProfiles: {
            type: 'object',
            defaultsTo: {}
        },

        toJSON: function () {
            var obj = this.toObject();
            delete obj.password;
            delete obj.socialProfiles;
            return obj;
        }
    },
    beforeUpdate: function (values, next) {
        CipherService.hashPassword(values);
        next();
    },
    beforeCreate: function (values, next) {
        CipherService.hashPassword(values);
        next();
    }
};

Here we’re adding a bunch of properties to our User schema, but also a method to remove sensitive data before converting our object to JSON.  We also have beforeUpdate and beforeCreate delegates defined to hash our password before saving to the data store.  You can see that the hash will use our CipherService, and one of the nice things about sails is that it makes our services available globally.  So here we can just use CipherService instead of needing a require. In Sails, you also have models available globally using their name. This is a huge advantage of Sails.

Auth Controller

Now we have to add an Auth endpoint to perform signup/signin etc…  We’re only generating a controller here, so you can just add the js file or do it the Sails way:

sails generate controller Auth

Now that we have our AuthController, let’s add the following actions:

/**
 * AuthController
 * @description :: Server-side logic for manage user's authorization
 */
var passport = require('passport');
/**
 * Triggers when user authenticates via passport
 * @param {Object} req Request object
 * @param {Object} res Response object
 * @param {Object} error Error object
 * @param {Object} user User profile
 * @param {Object} info Info if some error occurs
 * @private
 */
function _onPassportAuth(req, res, error, user, info) {
  if (error) return res.serverError(error);
  if (!user) return res.unauthorized(null, info && info.code, info && info.message);

  return res.ok({
    // TODO: replace with new type of cipher service
    token: CipherService.createToken(user),
    user: user
  });
}

module.exports = {
  /**
   * Sign up in system
   * @param {Object} req Request object
   * @param {Object} res Response object
   */
  signup: function (req, res) {
    User
      .create(_.omit(req.allParams(), 'id'))
      .then(function (user) {
        return {
          // TODO: replace with new type of cipher service
          token: CipherService.createToken(user),
          user: user
        };
      })
      .then(res.created)
      .catch(res.serverError);
  },

  /**
   * Sign in by local strategy in passport
   * @param {Object} req Request object
   * @param {Object} res Response object
   */
  signin: function (req, res) {
    passport.authenticate('local', 
      _onPassportAuth.bind(this, req, res))(req, res);
  },
};

So here we’re doing two basic things: Sign up and Sign in. The signup method uses the built-in Create method provided by the user model to create a new user. The signin used the Passport’s local authorization strategy to log in. In both cases, if it succeeds, we’ll generate a token so that they are signed in automatically. All requests that follow should now include the returned token in the header. We’ll demo this shortly, but have one final item to add first:  The login policy.

Policies

In Sails, the way to add Express middleware is via policies. If you examine the policy structure, that’s really exactly what it is. In our case, we need a policy that will protect our non-auth controllers from requests that don’t have a token. Look at the api/policies folder. There may be a sessionAuth.js file already present. You can delete this file as we won’t use it. Now add a new file to api/policies called isAuthenticated.js with the following contents:

/**
 * isAuthenticated
 * @description :: Policy to inject user in req via JSON Web Token
 */
var passport = require('passport');

module.exports = function (req, res, next) {
    passport.authenticate('jwt', function (error, user, info) {
      if (error) return res.serverError(error);
      if (!user) 
       return res.unauthorized(null, info && info.code, info && info.message);
     req.user = user;

     next();
    })(req, res);
};

The policy is pretty simple: It authenticates the user using the ‘jwt’ strategy that we earlier implemented in the passport configuration. If no token is present, this will return an unauthorized response. Otherwise it will add the user object from the token to the request.

Responses

Let’s briefly discuss Sails responses, another cool Sails convention. If you take a look at api/responses, you’ll see a bunch of responses that have been created for you already. These make it easy to define and reuse typical responses. So to return an OK response, instead of defining our response over and over, we can just do: return res.ok(data). In our case, we need to add two new custom responses: created and unauthorized.  In the responses folder add created.js with the following content:

/**
 * 201 (Created) Response
 * Successful creation occurred (via either POST or PUT).
 * Set the Location header to contain a link 
 * to the newly-created resource (on POST).
 * Response body content may or may not be present.
 */
module.exports = function (data, code, message, root) {
  var response = _.assign({
    code: code || 'CREATED',
    message: message 
       || 'The request has resulted in a new resource being created',
    data: data || {}
  }, root);

  this.req._sails.log.silly('Sent (201 CREATED)\n', response);

  this.res.status(201);
  this.res.jsonx(response);
};

Now create an unauthorized.js file in the same folder:

/**
 * 401 (Unauthorized) Response
 * Similar to 403 Forbidden.
 * Specifically for authentication failed or not yet provided.
 */
module.exports = function (data, code, message, root) {
  var response = _.assign({
    code: code || 'E_UNAUTHORIZED',
    message: message || 'Missing or invalid authentication token',
    data: data || {}
  }, root);

  this.req._sails.log.silly('Sent (401 UNAUTHORIZED)\n', response);

  this.res.status(401);
  this.res.jsonx(response);
};

I’ve also customized the other responses in accordance with some guidance provided by the Sails API Yeoman generator, but I’m not going to go through all of them.  If you would like to copy them, feel free to reference the project on GitHub. While you’re there, also checkout the overridden Blueprints in api/blueprints. These let you customize the standard processing of different operations exposed by the api controllers (unless you override them specifically in the controller). I also took these blueprints from the Sails API Yeoman generator.

Now we have our policy…how do we use it? In the /config folder, you should see a policies.js file. Open this file and adjust the code to the following:

module.exports.policies = {

    '*': ['isAuthenticated'],

    AuthController: {
        '*': true
    }
};

This is applying the following rules:

  1. Protect all controllers from unauthenticated users.
  2. Override this for the AuthController and allow anybody to hit that.

Loose Ends

OK, before we test this thing out, just a couple of small cleanups that will make Sails bug you less on “lift”. First open /config/models.js. Lets set up a standard migration policy so it won’t bug us on every project start.  Uncomment or add the line that reads migrate and change it to ‘drop‘ for now. This will drop the data store each time the application runs.  You can change to ‘alter‘ if you don’t want this behavior.

migrate: 'drop'

Now we’ll tell Sails not to expect a Gruntfile.  When you create Sails with –no-frontend, it doesn’t scaffold a Gruntfile, but it still warns that there isn’t one on every start. Go to /.sailssrc and remove the Grunt hook:

{
  "generators": {
    "modules": {}
  },
  "hooks":{
    "grunt":false
  }
}

Testing it Out!

Wow…that was a lot of stuff to run through but we’re basically done!  Let’s test it out. If you don’t already have Postman (or you’re preferred test tool) installed, go ahead and install it.

Now lets start up our Api.  At your project root terminal type:

sails lift

You should see sails start up on port 1337 by default. Now fire up Postman and signup a user by posting the following JSON payload to your local signup endpoint. In my case, this is: http://localhost:1337/auth/signup

{
 "username":"testdude",
 "email":"test1@test.com",
 "password":"testdude"
}

signup

The response should look like:

{
 "code": "CREATED",
 "message": "The request has been fulfilled and resulted in a new resource being created",
 "data": {
 "token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyIjp7InVzZXJuYW1lIjoidGVzdGR1ZGUiLCJlbWFpbCI6InRlc3QxQHRlc3QuY29tIiwiZmlyc3ROYW1lIjoiIiwibGFzdE5hbWUiOiIiLCJwaG90byI6IiIsImNyZWF0ZWRBdCI6IjIwMTUtMDQtMjRUMjE6NTg6MTkuMjcxWiIsInVwZGF0ZWRBdCI6IjIwMTUtMDQtMjRUMjE6NTg6MTkuMjcxWiIsImlkIjoyfSwiaWF0IjoxNDI5OTEyNjk5LCJleHAiOjE0Mjk5OTkwOTksImF1ZCI6Im5venVzLmNvbSIsImlzcyI6Im5venVzLmNvbSJ9.j9mSeoHJiNb_rzxqJ8Cefv5ctcMVzbgvnUlvAWhbXas",
 "user": {
 "username": "testdude",
 "email": "test1@test.com",
 "firstName": "",
 "lastName": "",
 "photo": "",
 "createdAt": "2015-04-24T21:58:19.271Z",
 "updatedAt": "2015-04-24T21:58:19.271Z",
 "id": 2
 }
 }
}

You’ll notice that a token was generated.  Copy the token that your API generated so that it’s available later.

We’ll now attempt to signin as well.  Post the following JSON to your local signin endpoint. In my case, this is: http://localhost:1337/auth/signin. Earlier in our Passport config we set up local auth to use email and password, but we could change this to use the username if that is preferable.

{
 "email":"test1@test.com",
 "password":"testdude"
}

In this case, the response should be almost identical but the response code will be a 200-OK instead of a 201-Created.

Now lets try to access a resource without our token. Perform a get on your local user endpoint. In my case: http://localhost:1337/user

You should get a response indicating that you are not authorized:

{
 "code": "E_UNAUTHORIZED",
 "message": "No auth token",
 "data": {}
}

Now lets update our request to add an Authorization header.  For the value, add “JWT“, then a space, then the token you created previously. So an example value would be:

JWT eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyIjp7InVzZXJuYW1lIjoidGVzdGR1ZGUiLCJlbWFpbCI6InRlc3QxQHRlc3QuY29tIiwiZmlyc3ROYW1lIjoiIiwibGFzdE5hbWUiOiIiLCJwaG90byI6IiIsImNyZWF0ZWRBdCI6IjIwMTUtMDQtMjRUMjE6NTg6MTkuMjcxWiIsInVwZGF0ZWRBdCI6IjIwMTUtMDQtMjRUMjE6NTg6MTkuMjcxWiIsImlkIjoyfSwiaWF0IjoxNDI5OTEzNTI3LCJleHAiOjE0Mjk5OTk5MjcsImF1ZCI6Im5venVzLmNvbSIsImlzcyI6Im5venVzLmNvbSJ9.9FqGAVnJNM3SWRupOqCoW7tPJqu0ChZt5f2_En6GKqo

getUsers

Now send the Get request and you should get back the user record we created when we signed up!

{
 "code": "OK",
 "message": "Operation is successfully executed",
 "data": [
 {
 "username": "testdude",
 "email": "test1@test.com",
 "firstName": "",
 "lastName": "",
 "photo": "",
 "createdAt": "2015-04-24T21:58:19.271Z",
 "updatedAt": "2015-04-24T21:58:19.271Z",
 "id": 2
 }
 ]
}

Final Thoughts

Seems like we did a lot of stuff here, but it was all pretty easy thanks to the conventions and code generation provided by Sails. In upcoming articles, we’ll look at expanding this example to include social auth, a front end via Aurelia and additional storage mechanisms (we’re just using disk here).

The code for this example is available here. The code in my uploaded GitHub example is using Postgresql instead of writing to disk, but that can easily be changed in the config/models.js file.  Just point to any connection you have set up in the config/connections.js file.

** Warnings **

Always remember that this style of token auth is not secure if the token can be intercepted. In production, always perform communications with the API (at least those that send the token or any sensitive information) over SSL.

Additionally, you don’t want to check any production secrets into public source control. In this case, the secret in the passport config file. In a follow up we’ll talk about how to handle this but you can read about config overrides here.

Nozus JS Preface: Sails with Passport and JWT – Ermahgerd!

The Sails project in this case is only going to serve as an RESTful API.  We’ll be using Aurelia on the front end, so we don’t need any presentation.  What we do need is authentication provided by the API.

So my local auth implementation went through a few stages and much hair pulling. I first took a look at sails-generate-auth. This seemed like a great starting place…but I ran into some issues:

  • You can use tokens to protect the API, but the social auth implementation is geared towards server emitted UI by default. I want something more SPA oriented.
  • It doesn’t use JWT currently although it supports bearer tokens.

So I moved on to Waterlock. Waterlock has a lot of cool features and is super-easy to set up. In addition, it appears to be more oriented towards a SPA/API implementation.

But I ran into some issues: It uses session in situations where tokens are being used.  This defeats some of the advantages of using tokens and reduces horizontal scalability.  Some remedies to this are being worked on, but have been stuck in pull request status for several months, although it looks as though they may be resolved soon.  So I figured I could just reference the github fork instead from npm until this gets resolved…but was still concerned about it using session.

And then with local auth in Waterlock: There is no register endpoint.  There is a login endpoint that auto-registers you if it can’t find you. 😦  So if you fat-finger your username it just goes ahead and makes a new user. Again there is a pull request that hasn’t yet been merged.

At this point I decided I needed to keep looking. I could try and get those pull requests across the line but I’m already leaning away because of the opacity/complexity of the library as a whole and the session requirement.  I don’t need something this complex but it’s a great reference implementation.

So next I found a Yeoman generator for Sails that was specialized for API’s and implemented with JWT based auth. This is a really cool project and major kudo’s to Eugene who was also very responsive to questions.  But then when I felt so close:  The Yeoman generator failed because of a Windows file system issue. I think probably a path length issue due to nested npm packages. I’ll see you in hell Windows file system!!!  Eugene mentioned that he hadn’t tested it on windows…so there it was.

This may have been a good thing because the generator/template also outputs a lot of functionality that is both very cool and yet unnecessary for my project. The upside being that you can blow out a complex API in almost no time, but I didn’t want to go through and attempt to clean out everything that I didn’t want/need.  I also wanted to go through the process myself so I understood it better.

So I used the generator templates as a reference project and borrowed some great things from it including updated API blueprints and a basic approach for issuing and validating JWT tokens.

So hopefully if you come across this post and are in a similar boat (npi), I can save you some of my frustration (aka “earned knowledge”). I’ll be adding the implementation information in the next post, but a reference implementation is already available in GitHub.  Hope that’s helpful!

Next in series:

Nozus JS 1: Intro to Sails with Passport and JWT (JSON Web Token) Auth

Nozus: Changing the API to Node JS and Sails

So after futzing around with ASP.Net 5/MVC 6 for a bit and becoming frustrated with attempting to implement token based auth, I decide to switch over to Node JS using Sails as an alternative for my API.

Why Sails?  I took a look at it a few months back and was impressed with the easy setup and Waterline ORM. I then had a recent job opportunity where the client was using Sails.  I passed on the opportunity but decided to review Sails again. The combination of convention with easy overrides is super productive and flexible.  Additionally, the documentation is well done for an OSS framework.

I look forward to the eventual rollout of ASP.Net 5, but it’s still under heavy dev and what works one day doesn’t on the next pull.  This is totally understandable but I’m ready to move along and get something working.  I’m already biting off a fair amount of learnin’ by attempting Node and Aurelia.  So without further ado, I’ve added the following growing list of articles on setting up Nozus in node, including examples in GitHub.

Nozus JS Preface: Sails with Passport and JWT – Ermahgerd!

Nozus JS 1: Intro to Sails with Passport and JWT (JSON Web Token) Auth

Setting up Babel with Gulp

So after initially using the WebStorm file watcher mechanism to transpile to ES5 using Babel, I decided to instead do it the “correct” way: using Gulp.  In this case my project is a Node/Express Rest Api, in which case I would end up using Gulp anyway for various other tasks.  Here’s the easy setup:

Add in the following dependencies using npm:

npm install gulp --save-dev
npm install gulp-babel --save-dev
npm install gulp-sourcemaps --save-dev
npm install require-dir --save-dev

My project structure is set up as:

  • src – The untranspiled source.
  • dist – The transpiled source.
  • build – The build files.  I typically put a paths file inside of build that has all of my project path information for conducting builds and add add a tasks directory under build for the actual build tasks.

We can now add a build.js file under build–>tasks which will look like:

var gulp = require("gulp");
var sourceMaps = require("gulp-sourcemaps");
var babel = require("gulp-babel");

gulp.task("build", function () {
    return gulp.src("src/**/*.js") //get all js files under the src
        .pipe(sourceMaps.init()) //initialize source mapping
        .pipe(babel()) //transpile
        .pipe(sourceMaps.write(".")) //write source maps
        .pipe(gulp.dest("dist")); //pipe to the destination folder
});

Now define your main gulpfile.js in the root project directory.  It simply uses require-dir to require all files in the build/tasks folder (to pull in all tasks).

require('require-dir')('build/tasks');

That’s it!  now run “gulp build” at the command prompt…all set.  This is obviously pretty bare-bones.  Normally I might also be using some other packages to set up gulp tasks that enhance my build process by:

  • Cleaning the dist directory pre-transpile (del)
  • Setting up a linter (jshint)
  • Running the project with change monitoring (gulp-nodemon)

Setting up Babel with WebStorm on Windows

FYI: this post refers to WebStorm 9.  Although the same approach should work with WebStorm 10, I found that WebStorm 10 already had a watcher set up for Babel.

So I decided to start working on a project combining Node and Aurelia, kind of a MEAN with Aurelia as the A.  I’m coming from a .Net background, so I’m on Windows.  I tried out both WebStorm and Sublime and was really drawn to WebStorm based on my familiarity with many of the shortcuts I’ve used forever in ReSharper.

So now I’m using WebStorm and I want to develop using es6.  WebStorm comes with a transpiler plugin (basically a file watcher) for Traceur.  I’m sure Traceur works great, but Babel has gotten a lot of good reviews and in addition Aurelia uses 6to5 (old Babel) out of the box, so why not stick with the same thing.  So I wanted to set up Babel as a custom file watcher in WebStorm…here’s the easy way to do that.

First, install babel via npm:

npm install babel -g 

So in order to run a WebStorm command, at least in windows, it has to be an exe, bat, or cmd file.  So add a new file to the root of your project and call it “runbabel.cmd” with the following contents:

babel %*

This tells Babel to run with any arguments passed in.  Make sure to **not** name the command babel.cmd as it will just call itself in a tight loop instead of calling the Babel CLI.

Now in the main menu, select File –> Settings…  From the resulting popup, go to Tools –> File Watchers and click the + button to add a new watcher.

Add file watcher

From the resulting modal, set up the watcher with the following settings:Create watcher

  • Name the watcher Babel (or whatever) and give it a description if you like.
  • Set the file type to JavaScript files.
  • Create a Scope and scope it to the directory containing your source files (in this project, that is src).
    • It’s important to set the scope properly because this is the directory that WebStorm watches, which is not necessarily the directory the program will operate on.  So initially I set this to be the project directory since the Babel already accepts a directory it should transpile, but I ran into issues because the watcher would get into a tight loop: Babel would output into a project directory which would retrigger the watcher, which would transpile, which would output new files and trigger the watcher again…infinity!   For more information on setting up a scope, check out the WebStorm docs on this subject.
  • For the Program select the runbabel.cmd that was created earlier, if you have it within your project, you can use the $ProjectFileDir$ macro to locate the command.
  • The Arguments can now be any arguments that the Babel CLI accepts.  In this case we’re saying that it should run on the src directory and output to the lib directory.

Now just select the Babel watcher you created…and let it rip!

SelectFileWatcher

Next, we’ll talk about how to set up the transpiler using gulp instead of a file watcher.