Websites are clients, too!

Historically, our API and our website have shared code and lived in the same binary (Scala, Lift), but in many other ways they were developed independently, with the API focusing primarily on the needs of our client teams. The site redesign and recently launched website features like the homepage map, lists, and notifications, have brought them closer together. With these features we’ve begun consuming our own public APIs, via JavaScript, directly from the website.

This strategy offers two important benefits. First, using the API directly from the client, we ensure that there’s only one code path for any given action. Second, it reinforces our commitment to keeping the API a first class citizen that is totally up-to-date. The deep link between the API and the website ensures they move forward in unison.

Crossing domains

To get there, we had to overcome the challenges associated with cross-domain (technically, cross sub-domain) communication between foursquare.com (web) and api.foursquare.com (API). Although we could theoretically make our API available on the same domain, doing so would undermine the security and production isolation benefits of our current setup.

Our API supports CORS, but not every browser we want to support does. To work around this we used a common technique in which an iframe hosted on api.foursquare.com is embedded on the foursquare.com web pages. The iframe executes a simple JavaScript statement setting document.domain to foursquare.com — notably the same as the web domain. Through the magic of the same-origin policy, this technique enables inter-frame communication while still allowing the iframe to make AJAX requests back to its original domain, in this case, api.foursquare.com. When we need to make api requests we can just use the iframe’s XMLHttpRequest object from foursquare.com pages.

<iframe onload="fourSq._tempIframeCallback()"
        src="https://api.foursquare.com/xdreceiver.html">
  <html>
    <head></head>
    <body>
      <script type="text/javascript">
        document.domain='foursquare.com'
      </script>
    </body>
  </html>
</iframe>

Getting some backbone

Then came to assembling the JavaScript code base around our API. To do this we’re taking advantage of jQuery, Backbone.js, Underscore.js, and the Closure Compiler.

We use the Backbone.js library to create model classes for the entities in foursquare (e.g. venues, checkins, users).  Backbone’s model classes provide a simple and lightweight mechanism to capture object data and state, complete with the semantics of classical inheritance.  As an example, fourSq.api.models.Venue is a structured JavaScript representation of the raw JSON returned by the venues API endpoint.

fourSq.api.models.Venue = fourSq.api.models.Model.extend({
  name: function() { return this.get('name'); },
  contact: function() { return this.get('contact'); },
  location: function() { return this.get('location'); },
  mayor: function() { return this.get('mayor'); },
  verified: function() { return this.get('verified'); },
  stats: function() { return this.get('stats'); },
  hereNow: function() { return this.get('hereNow'); },
  ...
});

From the code above, fourSq.api.models.Model is a simple subclass of Backbone.model that we use to provide some common convenience functionality across our code.  We also enumerate direct accessor methods for our fields as wrappers to Backbone’s regular attribute get method.  We find that this technique both removes extraneous syntax from our code and also serves to self-document the model schema, which in turn substantially simplifies maintenance, testing, and discovery. As an added benefit, it plays a little bit nicer with the closure compiler.

Instead of using Backbone.sync, we chose to write a complete service layer (fourSq.api.services) api that abstracts the underlying AJAX calls and raw argument/response types of the API.  The service layer allows us to enumerate the available APIs and funnel all service requests through a consistent code path.  Additionally, the service layer handles translating model objects to and from raw JSON.

Here’s an example of the implementation for the badges endpoint in the UserService.  Virtually all service endpoints have the same method signature; they take a map containing arguments that will be transferred in the underlying Ajax request, as well as success and error asynchronous callbacks.

badges: function(data, success, error) {
  var deserializer = function(data) {
    var sets = data.sets || {};
    var badges = data.badges || {};

    badges = _.map(badges, function(value) {
      return new fourSq.api.models.Badge(value);
    });

    return new fourSq.api.models.AnonModel({
      "sets" : sets,
      "badges" : badges
    });
  };

  fourSq.api.services.service_(
    fourSq.apiPaths.users.badges(data.id), data, {
      success: fourSq.api.services.wrapOnSuccess_(
                 deserializer, success, error),
      error: fourSq.api.services.wrapOnError_(error)
  });
}

In the code above, the success and error callback arguments are not passed directly to the underlying Ajax layer. The callbacks are wrapped to simplify the response object types that are ultimately received by the client code. The success callback, for example, is wrapped and paired with a function that deserializes the raw JSON response from the server into Backbone models keeping the client layer fully abstracted from the raw transfer format. The deserializer makes use of another helpful technique, the fourSq.api.models.AnonModel. The AnonModel mechanism helps keep our attribute access syntax consistent even in cases where we haven’t previously declared a model class – a function accessor is created for each key. In this example, result.badges() would return an array of badges.  (Oh yeah, and we’re using Underscore.js … it’s super amazingly helpful.)

Compiler? I hardly even knew ‘er

As we continue to grow our JavaScript codebase, we want to insure that we can iterate quickly while still developing modular, performant, and robust code. To help us reach these goals, we’ve integrated the Closure compiler tightly into our development environment. We run it in process and compile JavaScript on demand. Our code, whether served in production or during local development, is run through compilation that catches missing properties, typos, invalid method arguments, and much much more. The end result is not only minified code, but code that has been optimized. Lastly, since not every page needs all the javascript, we split the compiled output into modules that can be synchronously or asynchronously loaded as necessary.

Overall we’re happy with the first few features launched using this system, and plan on continuing to incorporate it going forward. If you like hacking on JavaScript and have a keen eye for awesome user interaction, we’re hiring!

– Mike Singleton (@msingleton), Matt Kamen (@losfumato), Dolapo Falola (@dolapo)