Tips by 60devs is a micro-donations platform that allows sending money (or tips) to people who helped you on the Web. The platform consists of a web app and a browser extension. The extension highlights registered users on Github, Gitter and Stackoverflow and the website of the service is the place where users can register and conduct payments to other users. In addition to the discovery mechanism via extensions, the service offers well-known old donate buttons which users can place wherever they want. Of course, there is a server behind the web app, the extension and buttons. tl; dr… koa-js-starter-project

And here is what the server is made of:

Server Side Technologies Used

The main technologies on the server are:

• nodejs
• redis
• pm2

We use koajs as a server framework and make use of generator functions almost everywhere. Also the following node modules are used:

"bluebird": "^2.9.30" for Promises
"co": "^4.1.0" for generator-based control flow
"co-redis": "^1.1.1" for Redis
"co-request": "^0.2.0" for request
"hat": "0.0.3" for id generation
"koa-bodyparser": "^1.3.0" for body parsing
"koa-cors": "0.0.14" for cors handling
"kroute": "^1.1.0" for routing
"winston": "^1.0.0" for logging

Our server is really simple and straight-forward thanks to generators.

ES6 Generators

I really recommend the following article - ES6 generators in depth by Dr. Axel Rauschmayer. It explains ES6 generators very well. And I just want to say that they allow you to call asynchronous pieces of code in a synchronous manner and that boosts your productivity a developer and makes the code clearer:

  var result = yield asyncJob(params);
  console.log(result);

asyncJob will do an asynchronous job and it will not block the loop but you will get the result into the result variable once the task is finished. So no callback hell anymore.

Server Code Structure

All code on the server belongs to one of three three types:

• middleware generator functions - these are handling HTTP requests. The define logic for different types of HTTP requests. For example, GET /user request is handled by one middleware function that defines a scenario for retrieving the data of the user identified by the authentication header
• shared generator functions - these are pieces that implement the shared logic. For example, a user management module that encapsulates how to retrieve/update user records in Redis.
• the start script - that’s the place where middleware functions are being assembled into a server. Basically, it defines how to map requests to middleware functions. We use kroute for this.

So this is how our server start script looks like:

// we refer to middleware functions by their filename thanks to `require-dir` module
var middleware = requireDir('./middleware');
var app = koa();
var router = kroute();

// POST request to /somePath maps to ./middleware/someAction.js
router.post('/somePath', middleware.someAction);
...
router.get('/somePath/:someVar/:someVar', middleware.someAction);

// allow these headers in cors requests
app.use(cors({ headers: 'accept, authorization, content-type' }));

// on error use ./middleware/onError.js
app.use(middleware.onError);

// also use ./middleware/authorizeUser.js to check users' authorization
app.use(middleware.authorizeUser);

// then parse body
app.use(bodyParser());

// then route to the middleware defined in `router`
app.use(router);
app.listen(4000);

We require all middleware functions and then map them to HTTP methods using the router. Additionally, we use koa-cors module to handle cors for us and koa-bodyparser to parse the body of incoming requests into JSON.

We had a problem because of the koa-bodyparser when we were processing PayPal IPN notifications. The problem was that we needed to access the raw body but it was already read and parsed by the body-parser and therefore it was not available again. The solution to this problem is to control when to use koa-body-parser:

app.use(function* (next) {
  var rules = [
      /\/paypal-ipn.*/
    ];

  var url = this.originalUrl;
  var isRawUrl = rules.some((exp) => {
    return exp.test(url);
  });

  if (isRawUrl) {
    // parsing the raw body
    this.request.body = yield getRawBody(this.req, {
      length: this.length,
      limit: '1mb',
      encoding: this.charset
    });
  }

  yield next;
});

koa-bodyparser checks this.request.body and if it’s NOT undefined it does nothing so what we had to do is to parse body before the koa-bodyparser. This is achieved by adding a koa middleware function before the koa-bodyparser:

  app.use(conditionalRawBodyParser); // for some special requests
  app.use(koaBodyParser); // for the rest of requests

We use environmental variables to configure the server for development and production environments. And we use pm2 to run and monitor the server in production.

Using PM2 on Ubuntu

The server start script has no logic for handling node processes or restarting them if they fail. PM2 is responsible for this. It’s quite easy to set up:

To start the server in the cluster mode running 4 processes:

pm2 start --node-args="--harmony" server.js -i 4

or in the fork mode (1 process):

pm2 start --node-args="--harmony" server.js

It’s important to add --harmony flag to enable generators.

PM2 can generate a startup script for you with:

pm2 startup

Now the server will be started automatically after the machine reboot.

Another important part is persisting the PM2 configuration using:

pm2 save

To stop/start/delete servers, use:

pm2 stop all
pm2 start all
pm2 delete all

Conclusion

So the server development (~10 different routes + some simple logic) took about 20 hours (development/testing/production setup). The toughest part was the integration with 3rd party services, i.e. Github, Stackoverflow and PayPal, because they have different APIs and the integration is harder to test. So far it’s my favorite stack for simple API servers: koa.js + pm2. Any suggestions of extendable libraries for easy integration with 3rd party websites are welcome!

Also I have created a starter project on Github that contains pretty much everything described in this post. In the next post will be about a cross-platform browser extension for the Tips service that we developed.

Thanks for reading!