Building a JSON API with Rails ? Part 4: Implementing Authentication

After reading the last post discussing authentication strategies, we now have a firm understanding on how we’re going to add authentication into our API. To recap – we’re going to use basic authentication for our initial username/password submission, and then token-based authentication on every subsequent request in which we just pass around a token to authenticate ourselves. We didn’t cover any code last time, but I promise it’ll be nothing but code this time.

Phase 1: The Initial Request

First off, we need to add a route that we can access in order to receive a token based on our submitted username and password. To do that, create this route in your routes.rb file:

To handle this route, we’re going to add a token action in our application_controller.rb. We’re putting it there because this logic doesn’t belong to any specific controller:

Simple so far, right? Now to add some actual logic to that action. Let’s update it with a handy rails method:

The authenticate_with_http_basic method is incredibly helpful, and really shows how Rails can help build an awesome API application. This method will parse the incoming request and look specifically for basic authentication information – which is set in the Authorization header. Not only does it automatically gather data from that header, but it will parse out the Base64 encoded username and password and return them to you as parameters inside of a block! How cool is that! As you can see above, I’ve appropriately named the two block parameters to represent this data.

Now if you try navigating to your /token endpoint, you’ll receive an error. That’s because the –api flag you used when you first created the project prevented many modules from being automatically included (since you often don’t need them in an API), such as the modules to handle the authenticate_with_http_basic method. You’ll need to include these modules in your application_controller.rb.

The first module is the one we need right now. The second module is included to handle an equally awesome token-based authentication method that we’ll use here in a bit.

Let’s finish out this token action:

That’s all we need to add to our token action. With this code, we are authenticating the user to verify they exist in our database and that the submitted password matches up with what we have stored for them. If so, we’ll return their token; otherwise, we return an error.

For the duration of this post, we’ll authenticate ourselves as a user with the username user@example.com and a password of password. If you included the seeds in your database that’s specified in the db/seeds.rb file we discussed in the very first post, then this user will already exist in your database.

Let’s make our first request to get this user’s token. First off, we need to get the Base64 encoded string of this user’s username and password. Open up your rails console and type in the following:

Now we can build our request and issue it with cURL:

This is a complete and valid request using basic authentication. If everything is set up properly, we should receive this back from the API:

Right off the bat it looks like we got an error, but everything’s working exactly as it’s supposed to. We just haven’t actually created any tokens for our users yet! By default, we want each user’s auth_token to be created when that user is created. To do that, we’ll need to update our user model:

Easy enough, right? Now when a new user is created, their auth_token will be randomly generated. However, the easiest way to make this happen for our existing users is to reset the database and let the seeds run again. To do that, run:

After the database is reset, we can rerun our initial request to get a valid token:

And bam! We got our user’s token! Keep in mind, your token will not be this exact same one since it’s randomized, but it will be in a similar format. Now we can build our token-based authentication, and feel safe knowing that we’ll never need to include our personal password in a request again.

Phase 2: Handling Every Other Request

We now have our token for the user that we’re authenticating as. Since we’ll be using this token on every subsequent request to this API, you’ll want to store it in some storage structure like a cookie, session storage, local storage, etc. Now let’s say we want to make a GET request to /posts/1 to receive data about the first post. Keeping in mind the token-based authentication format that we discussed in the previous post, we will build our request like so:

In fact, if you make that request right now, it will go through – but that’s because we haven’t built any authentication yet! We want to prevent any resources from being accessed unless the requestor is properly authenticated. To add in the handlers for this authentication, we will again be editing our application_controller.rb:

We are adding a before_filter hook that will call our created authenticate_user_from_token method on every single request, except when the user is requesting the initial token (since they don’t know their token yet at that point). Let’s update that authenticate_user_from_token method now:

And this is actually all we need to add. Remember how we used a fancy authenticate_with_http_basic method in our token action to handle basic authentication? We’re using a similar method here in this hook to handle token-based authentication. The authenticate_with_http_token method will look for an incoming request and parse the Authorization header again, but in this case, it looks specifically for a token-based authentication format. We only pass in one value with this form of authentication (which is the token), and you can see above how this method will parse out our token and provide it as a block parameter. We additionally also receive an options parameter, but we won’t be using that.

The logic that we added in our authenticate_user_from_token method will parse an incoming request and validate not only that it is using token-based authentication, but that the token corresponds to an actual user. If the token is valid, then the request continues as normal to /posts/1; if the token is invalid (or completely missing), then we will receive an error.

As an example, if we submit this request again that we did earlier:

It will work perfectly and return the first post’s data. But if we change up the token just a little bit and remove that last character like so:

Then we will receive the following error:

And that’s it! You now have a pretty secure API with all the benefits of token-based authentication (don’t remember those benefits? Review them in the last post). For debugging purposes it’s often a pain to have to worry about authentication, and I kept that in mind as I was building this architecture. If you ever want to make a request to a resource without having to authenticate, then just comment out the before_filter line:

And now all of your requests will go through without worrying about authentication. Just remember to turn it back on before you push anything to production!

Finale

You officially now have a thorough base API with a lot of the major concerns hammered out. This concludes the 3 major points that I wanted to discuss – scaffolding an API, serialization, and authentication. But, I still have a couple more parts I want to cover like writing tests for an API as well as a general overview of some other API topics such as Rails vs Rails API file structure, nested vs. flat routes, CORS, and filtering resources based on query parameters – so don’t think we’re quite done yet.

You can check out all those smaller concepts in the next post in this series – Afterthoughts!


P.S. If you want to see an example JSON API built with Rails using everything that we’ve discussed so far, check out my example API GitHub repo based on a talk I gave at a local Ruby meetup.

Building a JSON API with Rails ? Part 3: Authentication Strategies

We just wrapped up our discussion on serialization in the last post, and technically at this point you have a fully functioning base API and don’t need to do any more work (if you’ve been following along in this series). However, there’s one caveat – we don’t have any security. So far anyone can request data from any endpoint without considering data that’s private to certain users or data that requires certain privileges to view. That may be okay if you’re building a completely public API where any user can make any type of request at any point in time, such as a public wiki, but chances are that you need to put at least a little bit of security in there.

And if you still aren’t biting, wouldn’t you like to at least know which user is actually making the requests, so that you can tailor some of the data you return to them? You can’t do that right now, but you’ll be able to by the end of this topic. Let’s begin.

Some Background

Before we get into adding code, we need to discuss how different authentication strategies work according to the HTTP spec and what options are available to us. You can always use a third-party authentication package like OAuth, which provides a very high-calibur authentication system that is completely free to use – but a lot of these systems are a tad bit complicated to implement. Should we be building a massive app, this may be something to look into, but since we’re building a small blog AND the main purpose here is to educate, we’re going to build our own simple yet very powerful authentication system.

We’re going to use two different authentication methodologies in this app: Basic authentication and Token-Based authentication. Why use two? Because we’re using them for different purposes. Check it out.

Basic Authentication

Basic authentication is what we’re going to use to accept a username and password from the user in order to make the initial authentication request to our application. Whenever you log into any application that asks for a username and password, chances are that they’re using basic authentication when you make that initial login. Basic authentication is actually a part of the HTTP 1.1 spec under rfc 2617, and thus has a specified format that we can use. It looks like this:

We’re doing a few things here:

  • Setting an HTTP Header called Authorization
  • Submitting the value of that header to be “Basic username:password”

HTTP requests can have headers such as Content-Type, Host, User-Agent, Cookie, etc. and the one we use to authenticate is called Authorization. The first part of our header is the word Basic; this is how we state that we’re using basic authentication instead of a different form of authentication. The second part of our header is the concatenation of our username and password, separated by a colon. But, I left out one major part: Our username and password chunk isn’t in plain text – it’s Base64 encoded. What does that look like? Here’s an example with username foo and password barbaz:

So after it’s all said and done, our full Authorization request header would look like this:

Now you might ask, why do we Base64 encode the username and password? Your immediate thought might be to add security so that our password isn’t in plain text, and while it sure looks that way, that’s not true. We eventually need to decode that username and password block on the server, and there’s no secret on how to do that – so if anyone gets a hold of that Base64 encoded block while the request is being made, they can decode it in a heartbeat. The real reason Base64 encoding is employed is to make the request header fully URL safe. There are a lot of characters that aren’t URL safe that we use in our passwords and perhaps even in our usernames, and Base64 encoding ensures that all of that gets encoded into a string that is fully HTTP transferable.

Back to our app. As I mentioned earlier, we’re going to use basic authentication to submit our username  and password to our API, and it will return a token to us – a token being just a random hash of letters and/or numbers. So what good does that do us? It does us a lot of good because now we can use token-based authentication.

Token-Based Authentication

We received a token back from our initial request using basic authentication, and now we’re going to use that token on every subsequent request that we make to our API to authenticate ourselves. We do this because it’s actually a core principle of Representational State Transfer (REST) for the server to not maintain any concept of state, so therefore each unique request is responsible for providing all of the necessary authentication data every single time. Will this be ridiculous to implement? No, it won’t be bad at all – it just means that we’re going to set another header on every normal request we make to our API. Although token-based authentication isn’t specified in the HTTP spec like basic authentication, there is a very agreed upon format to structure your token-based Authorization header that looks very similar to basic authentication:

See? Very similar structure, and this one’s even a little simpler because instead of providing two values, we only have to supply one: our token. Now remember, our token is just a random hash of letters and numbers that don’t mean anything – so why do we use it? The idea behind passing a token around on every request is that it is a way to represent the user making the request without having to pass around any important data. On our server, we would associate a user with a certain token (if you remember back in part 1, this is when we’ll actually use the auth_token attribute we created on our User table); because our server is aware of this association, we can authenticate that user solely based on his/her token. That’s using just one value to authenticate, instead of having to pass around a username and password on every request.

Let’s say that we submit our username and password using basic authentication, and our server responds with a 200 (i.e. successful) status code and the following JSON:

That means we got our token! We can now change our Authorization header to use that token for every normal request we make to our API:

And this will handle both authorizing us as a valid user and it will let the server know who the requesting user is so that we don’t have to worry about explicitly telling the server through query parameters or other headers.

Worst Case Scenario

What happens if that token is compromised? Well the only way that could really happen is if someone is listening to the requests you make via some shared connection like public wifi. If a black-hat hacker does get a hold of your token, then they can make a few requests – but as soon as it’s discovered that you’ve been hacked, then we just reset your token. That would require you, the rightful user, to have to log in again, but that’s all you have to do – the server handles issuing you a new token and authenticating with that one. The hacker that has your old token, he can’t do anything with it now. That’s much nicer than having a hacker compromise your actual username and/or password, because:

  • You may use that username and/or password on multiple sites
  • You would need to manually change your password if the hacker compromised it

Sounds like a pain. This is why token-based authentication has really gained traction over the past several years.


No Code?

We didn’t touch on any code here, and I apologize for that. But I needed to make sure we covered these authentication strategies before we actually start implementing them so that you know why and how we’re going to use them. As I mentioned earlier, OAuth is an example of a great third-party authentication package that takes token-based authentication to the max – but for the sake of simplicity and education, we’re going to build our own strong auth into our API.

So get ready for the next post where we’ll be doing nothing but code. We got through the learning part here, now we can move to the fun stuff. Rails provides some seriously awesome support for these authentication strategies, and now that we know how they work, we can push the power of Rails as an API to the absolute max.

Part 4: Implementing Authenticatation

Building a JSON API with Rails ? Part 2: Serialization

Welcome to part 2 of our API building adventure. If you haven’t read Part 1: Getting Started yet, then I highly recommend you go through that post real quick to make sure we’re all on the same page. We’ll be continuing to develop on our Blog API which uses three relational tables: User, Post, and Comment. In the last post, we focused on setting up a basic JSON API using Rails, preparing our database, and reviewing how to issue requests to that API. Today, we’re going to take the power of our API to another level with serialization.

Serialization

So what exactly does it mean to serialize our API? Currently when we make a GET request to one of our API endpoints (like /users/1 or /posts/1), we get back all of that object’s attributes from the database record (or multiple objects’ attributes, if querying an index action). This might seem okay at first, but let me give you an example of how this is undesirable. When we query /users/1, we will get all of that user’s data – including the unencrypted password. That’s a huge security flaw. Additionally, what if we wanted to query that same endpoint and return each of that user’s posts in addition to their user attributes? We can’t do that right now. That’s where serialization will help us.

We employ serialization in APIs to properly handle the response from our GET requests to the API and format the data exactly how we want. Could we handle this directly in the controllers? Yes, but that gets real messy real quick, and it all goes back to our concept of SOA – Service Oriented Architecture. Controllers are meant to handle the business logic of our app, and not focus on formatting a response – but handling response data is exactly what serializers are for, so let’s use them and keep our API clean and modular!

Setting It Up

There are different ways we can apply serialization in Rails, and all of them involve gems. The three most common serialization gems are:

Each of these gems are very well supported and have hundreds of forks on GitHub, but I prefer to use ActiveModelSerializers (AMS) – predominantly because right out of the box it plays very nicely with Ember.js via Ember Data’s ActiveModel Adapter. If you’ve used Ember, then you know it’s very powerful, but you have to play by its rules – and using AMS allows you to do that. If you don’t use Ember, AMS is still a wonderful serializer and is very Rails-esque in syntax.

Let’s install AMS by adding it to our Gemfile:

Then run a bundle install. AMS comes built-in with generators, so to create a serializer for our User resource for instance, we just run:

And that will create the following file:

Now, if you navigate to your /users URL, you should see JSON that looks like this:

This is different from what we’ve seen in two ways.

  1. We now have a root users key and are returning a JSON object instead of an array of JSON objects.
  2. We are only rendering the id, and no other data on the User objects. That means no more exposed passwords!

See how simple that was? This same serialization pattern will also carry over for all of your controller actions that handle GET requests that return JSON. Now go ahead and run the serializers for the remaining Post and Comment resources, and then we’ll get into some configuration:

Configuring the Serializers

We won’t go into full configuration options here, as you’re better off checking the AMS documentation for that, but we’ll go into the core options that will help you the most. If you want to return more model fields than just your ID, then you just need to add them to the attributes method call like so:

And now when you query your User endpoints, you’ll receive the email and created_at fields too – easy as pie! But that’s not all. Let’s say you wanted to query a User endpoint and return each of those user’s posts too. Well that’s easy, and here’s where you really see the Rails-y design of AMS:

And wallah! You are now returning each user’s posts when you query a user – and the JSON data for each post will also follow the configuration in the serializer created for the Post resource.

I just have one last serializer configuration I wanted to share. Occasionally, you may want to modify the data that you return in JSON, but because this specific alteration is only meant for serialization cases, you don’t want to dirty up the model files by creating a model method. AMS provides a solution for that. You can create a method inside your serializer and therein access the current object being serialized, and then call that method with the same syntax as if it were an attribute on that object. Doesn’t make sense? Take a look at this example:

Now our serializer would spit out a say_hello JSON key that would have as its value the word “Hello” followed by that user’s email address. You access the current serialized object via the ‘object’ variable inside of any method you define inside your serializer. Nifty, huh? Pro Tip: You can also add model methods into your attributes method call, and don’t have to redefine them in the serializer.

One last thing (didn’t I already say that above?): If you don’t like the JSON syntax of having a root key like user or whatever resource you’re querying, then you can go back to the old syntax we had where it strictly returns either just the JSON representation of the object (e.g. show), or an array of JSON objects (e.g. index). You just have to add a short method in your application_controller.rb to set it globally:

Wrap Up

That’s it for serializer configuration that we’re going to cover in this post, but there’s a lot of other neat options you can play with using AMS. As I mentioned earlier, I initially chose AMS over other serialization gems because of how nicely it plays with Ember.js, but it’s built to be completely agnostic of whatever front-end framework you use. For example, I’m currently working on a large Angular.js app, and AMS is still my chosen serialization gem of choice because it does everything I need it to (and beautifully at that).

We’ve now covered the actual building of an API and serializing our JSON response to format it exactly like we want. Technically this is all you need in your server-side API, but I want to review one more very important topic: Authentication. After all, there’s a big chance that you plan to build an API that houses personal data that you don’t want everybody in the whole world to be able to query. In the next post, we’ll cover how to authenticate your requests so that only you can access your personal data, and no one else can!

Continue on in part 3 of this series: Authentication Strategies.

Building a JSON API with Rails – Part 1: Getting Started

How can you go about supporting your Angular/Ember/Backbone app, iOS app, Android app, Windows phone app, etc., all with a single back-end? Here?s how: by breaking up your backend server into its own API service, and Rails is just the framework to help make it simple and awesome.

This service separation is dubbed Service Oriented Architecture (SOA), and by following it and building a JSON API that’s separate from the front-end, you’ll be able to support all of your related apps with this single service and keep your back-end incredibly simple, logical, and easily maintainable.

In order to build a solid API, we’re going to break our goal up into 3 different tasks:

  • Build out our models and our data
  • Serialize our data
  • Add Authentication

Adding authentication is always an optional step, but most of the time you’re going to want to prevent unauthorized users from accessing your API. We’re going to tackle just the first task in this post, and save the rest for the next two parts. Ready to start? Good.

Starting Out

First off, we install rails similarly to how you’ve always done it – but with the new Rails 5 API flag.

As you can probably see, we’re going to be creating a blog, and it will have the following database structure:

  • User has many Posts
  • User has many Comments
  • Posts has many Comments

Building the API

Just a simple 3-table database will suffice, so let’s use the rails generators to scaffold out our 3 models:

If you’re familiar with Rails, then this looks very familiar to you. In fact, you can probably tell very easily what each attribute’s purpose is – except for auth_token perhaps. We’re going to make use of that attribute later on when we discuss and build in authentication, so don’t worry about it for now.

After you migrate your database, then you have a fully functioning API! Start up your local rails server and navigate to

to see your API live. It should look like an empty array set, which is what we want – just pure JSON.

Seeding the Database

If you want to add some seeds to populate your database, you can add this code to your db/seeds.rb file:

Now, all you need to do to run these seeds is update your User and Post model files with the necessary has_many relationships like so:

And then run the seed command to pre-populate your database:

Now your database has real data!

Issuing Requests

Because we scaffolded our resources, we created controllers that are fully capable of handling the standard HTTP requests types: GETPOSTPUT/PATCH, and DELETE. If you’re a little unfamiliar with these names, you can map it to the common CRUD acronym:

  • Create (POST)
  • Read (GET)
  • Update (PUT/PATCH)
  • Delete (DELETE)

The URLs for issuing any of these requests are:

You can obviously issue GET requests by visiting these pages in your browser, or you can use the curl command from your terminal (or similar command) to issue any of these requests. Everything will work as expected.

That’s it?

No, of course that’s not it, but look at what we’ve done so far – we’ve built a relational database with a fully functioning JSON API on top that can handle any of the 4 main request types, and we did it in practically no time flat. We have a lot more to talk about such as serialization, authentication, and an overview post discussing some of the bigger questions that come up when you’re building an API, so if you’re ready, feel free to move onto part 2.

Happy API Building!