You’ll Never Be the Best Developer

And that’s okay. You don’t want to be.


Let’s be honest; developers have a really great career potential compared to the effort it takes to become one. The salary is strong right out of the gate, and it increases pretty quickly compared to other professions. We don’t even need college anymore to be classified as a talented developer – and the rise of bootcamp classes and self-education courses have helped introduce a new form of education that programming seems to be pioneering. Does that mean college isn’t useful? Absolutely not – but do the cons (cost, length of time) outweigh the pros? Well, that’s a good question that I can’t answer.

As good as our careers are though – it’s nothing if not competitive. Is it tough getting a job? In a reasonably sized metro area, no it’s not – and working in more rural areas isn’t difficult either if you’re able to have solid internet access and are open to remote opportunities. When I say competitive, I mean mentally competitive. There’s always something new to learn – and that’s honestly a good thing; it prevents developer burnout and keeps us “hungry” for learning more. But it’s also taxing to think that we always need to be sharpening our skillset every single day. Not only do we have new languages, frameworks, tools, etc. coming out daily that we can learn, but when we see other developers (it doesn’t matter if you know them) using these tools and building cool things, there’s usually an accompanying thought of “man, I wish I could do that.” And then we start comparing ourselves to other developers in a fashion that already places us at a disadvantage, and that’s not healthy. Social media isn’t helping with any of this either; when we see our fellow devs retweet cool things, or we see the next hot dev thing on Reddit, most of us will have bittersweet feelings; it’s awesome to see our community progressing, but it’s just another thing that someone else can do that you can’t.

That’s where the title of this post comes into play. You’ll never be the best developer. And you know it too, deep down.


The best programmer in the world is someone you’ve never heard of. They’re not the people who speak at conferences, or lead the CERN or NASA dev teams. They don’t tweet about what they do, market their work on GitHub, or communicate in IRC or slack channels. They don’t have time to do that, because to be the best, they can’t stop – because if they stop, that means someone’s getting better. If someone’s getting better, then they’re not the best – so they keep going. It’s not healthy, and that’s what it means to be the best. Look at the athletes who have been dubbed #1 in the world at some point in time, and chances are they had severe physical issues later in life compared to the average joe. Think CEOs are the best? The divorce, suicide, and incarceration rates there are astounding. You don’t want to be the best.

Back to developers now; so who is the best developer in the world? Here’s the raw truth: he/she doesn’t exist. There is no best. Confused yet? Being the best is just a concept in our career (and our career isn’t the only one). There will always be someone better than you at something, and there’s absolutely nothing we can do about that.


So what are we supposed to do if we’ll never be the best?

You stop trying, that’s what you do. Do you stop growing as a developer, or trying to learn new things? Absolutely not – but let go of the thought that you need to be the best, because it’s never going to happen. Don’t even think about being better than someone else. Just aim each day to be a little bit better than you were the day before (and it doesn’t have to mean coding-wise). One of my favorite quotes seems very applicable here: “Every man I meet is my superior in some way, and in that I learn from him” (Emerson). Learn from everyone, because you can.

You’ve got a family, and friends, and non-coding hobbies. Go enjoy life, and stop worrying about being the best. Just try to be better instead. We can all do that.

Declarative Programming with Prolog – Part 3: Putting it All Together

Welcome back to the final post in this Prolog series! If you haven’t done so already, please check out the first two posts before starting this one – or else it may not make much sense. Today, we’re gonna take what we’ve learned so far and build a real live application to showcase how powerful Prolog can be. We’ll be building a sudoku solver, which will be a Prolog script that accepts an unfinished grid and will solve the rest of it on its own. Here’s the crazy part: did you know you can do this in just 15 lines of Prolog? We’ll be showing that minified script later in this post after we go over a simpler way to build our solver, as well as other powerful ways that you can use Prolog. You’ve come this far in the series – let’s finish strong!

What We’re Gonna Build

Unfinished Sudoku GridWe’re gonna be building a sudoku solver, and just to make sure we’re all on the same page, let’s briefly review how the game of sudoku works. In sudoku, you’re provided an unfinished 9×9 grid that you have to complete, and each cell must be a number between 1 and 9. The catch is that each row, cell, and 3×3 square of the grid must contain the numbers 1 through 9 and they must be unique (e.g. you can’t have the number 1 twice in the same row).

For the duration of this post, we’ll be building a script that solves a 4×4 sudoku grid, instead of a 9×9. The concepts will all be the same – but the smaller the grid size, the easier it will be to both explain and to grasp the concepts.

The Full Script

Before we get deep into solving this problem – I want to share the full Prolog script as well as the query we’ll be using:

Note: I take zero credit for this code; it comes from the Pragmatic Bookshelf book: 7 Languages in 7 weeks (although it was slightly modified to work with SWI-prolog instead of GNU Prolog).

And our query:

Our solved sudoku board will get unified to the Solution variable, which will print out when we run this query. Even without a deep explanation – if you look at this script for a minute, you can probably grasp most of the concepts; don’t worry though, we’ll still explain everything!

Building Our 4×4 Sudoku Solver

First off, we need to include the clpfd module because we’ll be using a couple of pre-defined predicates in our script. I’ll bring attention to these when we use them.

Next, we start off our sudoku rule which accepts a list with a length of 16 (i.e. our unfinished sudoku board), as well as a variable which we’ll unify to our solution:

We then unify our Solution to our list, and then further unify our list into 16 other variables – one to represent each cell:

This next line is our first predicate from the clpfd module that we’ll be using: ins. This tells our Puzzle variable that we only want to set its entries to values between 1 and 4 – and since our Puzzle variable was unified to the 16 variables representing each cell, this logic caries over to those variables as well.

The next 14 lines are just more unifications. We’re creating variables to represent each row, column, and square, and setting them each equal to a list with a length of 4 to represent their cells.

So far all we’ve done is some simple unification; we haven’t written any actual logic that tells Prolog how the rules of sudoku work. Well, believe it or not, that’s the easiest part! That’ll only take 6 lines to accomplish, and it will complete our script:

Here in the final subgoal of the sudoku rule, we’re querying another predicate (the valid predicate) and passing in a list containing each row, column, and square. Now we need to define what the valid predicate actually entails – but wait, there’s two of them up there! Yup – that’s because we’ll be using recursion similarly to how we did in the last post.

Here’s a list of the actions that take place when we first call the valid predicate:

  • the first valid predicate is called – which is a fact. This is our base case.
  • This fact returns false because the list that was passed in has 12 entries (4 rows, 4 columns, and 4 squares), and the fact is trying to unify it to an empty list – which doesn’t work.
  • Next, the valid rule is called, which immediately splits the passed-in list into a Head variable which contains the first entry in the list (which is the first row), and a Tail variable which contains the remainder of the list (everything but the first row).
  • The first subgoal of the valid rule calls the all_different predicate, which is the other predicate we’re using from the clpdf module. This predicate simply accepts a list as an argument and checks if each entry in the list is unique. Nothing crazy.
  • Our last subgoal of the valid rule is a tail-optimized call back to the valid predicate – BUT – we’re passing in the original list minus its first entry.
  • This process repeats – and each time the stack frame changes, valid is called with one less length.
  • Eventually, after 12 loops, the first valid fact (our base case) will return true because at that point, the list will be empty.
  • This completes the entire query.

After the valid query completes, Prolog will have properly unified our Solution variable to either a valid solution, or it will have determined that a solution wasn’t possible. Let’s see what we get when we issue our query:

We did it! We have a valid solution! And while we only solved a 4×4 grid, these exact same concepts apply if we wanted to solve a 9×9 grid instead.

Why This Is Awesome

We just solved a 4×4 Sudoku grid using Prolog – but if you’re not impressed yet, then you’re bound to be asking something like “I can build this in an imperative language in a similar way – why is this impressive?” Well, think about it. If you wanted to build this logic in an imperative language, then you could certainly create a function that accepted a list of a completed sudoku board and use this same logic to verify that the board was indeed a valid solution – but there’s no way that an imperative language could actually solve the board on its own without a lot of extra logic that either you or some other module would provide. See – this is what makes Prolog (and declarative programming) so powerful; we never told Prolog how to solve the board – we just gave it the rules! We just told it that each row, column, and square had to be unique – and that’s it! It figured everything else out on its own!! You just can’t do this type of thing in imperative languages!

This is the power of unification in Prolog – and if you haven’t understood how it’s fundamentally different from assignment in other languages, then I hope this helped to paint that picture. It’s these types of problems that Prolog is meant to help solve.

Now, I mentioned in the intro that there are sudoku solvers written in 15 lines, and while we won’t be getting into the nitty-gritty of how they work, it’s safe to say that they’re based off of the same concepts that we discussed in our little example above. Here’s a demo of a real sudoku solver in Prolog; this solution also solves the full 9×9 puzzles – not the little 4×4 puzzle we solved above:

And our query:

Bam – look at that!

Taking Prolog a Step Further

Hopefully you’re impressed with Prolog by now – but maybe you’re wondering how you would interact with Prolog from another program. Surely not via issuing command line queries, right? Well, how about an HTTP Server?

Yeah – Prolog can do that too!

Prolog Server Response - Hello World!

Amazing, right?!

The Series Conclusion

There will always be problems that imperative languages can solve better than declarative ones; after all, there’s a reason why they dominate the programming market. But, that doesn’t mean that declarative languages don’t have their place in the programming world – for as you saw throughout this series, you can do some really cool, powerful things with Prolog.

The big takeaway is this: Using Prolog where it’s not meant to be used would be silly – but by using it in a situation where it can really shine, you’ll really turn some developer heads. And regardless of if you ever do use Prolog for fun or in your career – at the very least, I hope this helped broaden your perspective on different programming paradigms by teaching you a little bit about declarative programming.

Thanks for sticking with me this far! Feel free to check out the whole GitHub repo complete with all of the demos that we coded throughout this series.

Declarative Programming with Prolog – Part 2: Unification, Recursion, and Lists

Welcome back to this series on Declarative Programming with Prolog! If you haven’t already, make sure you check out the first post about getting started with Prolog, because in this post we’re going to dig deeper into some core concepts – specifically unification, recursion, and lists. Without further ado, let’s get into it.

Unification

We’ve already briefly mentioned unification and how we used it to set our variables in our rules and queries – but there’s more to it than that. At its core, unification means to make everything equal a valid result. This is different from assignment because we’re not usually telling Prolog exactly what our variables should equal – it’s figuring that out on its own.

Let’s use this example to understand more about unification:

The first two lines are nothing new – they’re just facts. We can even understand the next rule by now – especially now that you know unification is happening under the hood. When we query failed_houses with two variables, Prolog is unifying those variables to values that fit all of the subgoals – and as you can see, there are multiple combinations of values that satisfy them all:

Let’s take a look at that last rule now: little_pigs. What’s happening here is that we’re unifying the variables that are passed in to predetermined atoms – respectively straw, wood, and brick. This means that the rule will succeed only if we either pass all arguments as variables, or if we pass in arguments as atoms that already match those preset values:

However, the rule will fail if we try to pass in an atom that doesn’t match what we’re unifying the variables to:

Unification is a cool concept that’s core to how Prolog does its magic. The concepts below (as well as the last post in this series) will really help to illustrate unification more – keep following along!

Recursion

Just like in many functional languages, Prolog doesn’t support formal looping constructs like for or while loops. This might seem like a limitation at first – but fear not; Prolog handles these needs with some snazzy recursion.

We’ll use this example to illustrate recursion in Prolog:

In this example, henry is a boss to rob, who is a boss to john, who is a boss to sam. We’re going to use recursion to illustrate a relationship between these 4 individuals beyond what we can query via the facts. To do this, we’re going to use two rules called higherup – each that accepts two arguments. It is completely valid to have facts and rules that have the same predicate and arity in Prolog – and only one of them ever needs to return true for the whole query to return true.

The first higherup rule is a base case that just checks if the two arguments have a direct boss fact that links them together.

The second higherup rule is where the real recursion happens; it accepts two arguments and tries to satisfy two subgoals: one which checks if X is a boss to someone, and another that checks if that same someone is a boss to Y. Confused yet? Let’s do an example

This query to the higherup rule will check the first rule it finds – which is our base case that checks if the two variables match to one of the boss facts. In this case, they do! The rule returns successfully, which means the whole query is true since only one successful rule or fact per predicate is needed. It never queries that second rule.

That was a pretty boring example – so let’s do another one:

Here is where the real recursion happens, and I’ll list out a chain of events to explain exactly what’s happening here:

  1. Prolog queries the higherup rule with the atoms henry and sam. The order matters.
  2. The first higherup rule fails because there’s no direct boss fact for henry and sam.
  3. The second higherup rule is queried:
    • The first subgoal unifies X to henry, and (we assume) Z to rob, since that’s currently the only atom with a common fact to henry in the order the arguments were passed.
    • The second subgoal queries higherup again – this time passing in rob as the first argument and sam as the second.
  4. The first higherup rule fails again because there’s no direct boss fact for rob and sam.
  5. The second higherup rule is queried:
    • The first subgoal unifies X to rob, and (we assume) Z to john, since that’s currently the only atom with a common fact to rob in the order the arguments were passed.
    • The second subgoal queries higherup again – this time passing in john as the first argument and sam as the second.
  6. The first higherup rule passes this time, because there IS a direct boss fact for john and sam!
  7. The whole query returns true.

Overall, it took three loops over our rules for our query to return successfully. You should be careful with recursion because as with most other languages, you can easily run out of application memory if you loop too many times. That is, unless you tail-call optimize your recursion, like we did in this example.

Tail-Call Optimization

Prolog (among many other languages) has the power of tail-call optimization during recursion which means that it’s possible to maintain constant memory usage throughout – no matter how many times you loop. You could loop 2 or 200 times – and memory usage would be the same. How is this possible, you might ask? This happens when your recursive action is the very last thing called in your rule (i.e. the tail call). In our example above, the very last subgoal of our very last rule with the higherup predicate is where our recursion happens – and every time it gets to this subgoal, Prolog only needs to remember one thing about the current stack frame: Is the query true up until this point? No logic will happen in the current stack frame after the recursive call – so it doesn’t need to store any data about the current stack frame because there’s no point in doing so. Each time Prolog comes across the recursive call, it knows that it can keep going deeper and all it has to remember is that the query is true up until that point. That means that when it finally does satisfy a base case or return false – it doesn’t pop all the way up the stack chain. It just flat out returns true or false right from where the query ended.

All of this goes out the window if you place another subgoal after the recursive call, or if you place another rule/fact with the same predicate and arity after the rule with the recursive action – because in that case, Prolog would need to store the state of the current stack frame so that it could resume logic at that point. For tail-call optimization to work, the recursive action has to be the last possible action in the current stack frame.

Lists

Lastly, we need to review how lists work. In Prolog, lists are containers of variable length, while tuples are containers of fixed length. Lists are indicated by brackets ( [] ) while tuples are indicated by parentheses ( () ).

While both certainly have their use-cases, we’ll focus solely on lists in this example because they can do some neat things that tuples can’t.

We can unify values inside of lists (as well as tuples):

You’ll notice that the variables are on the other side of the equals sign. This is probably weird to see – but remember that unification is different than assignment. Prolog uses unification to make sure that everything matches to a valid value – no matter where it is. We can even spread the variables on both sides of the equals sign:

Taking things a step further, we can deconstruct a list by splitting it into a head entry and another list that makes up everything but that first head entry:

This is something that tuples can’t do – and this becomes a really powerful concept behind what all you can do with lists. You can deconstruct a tail list even further:

In fact, if you wanted to get the n-th entry of a list, you would use list deconstruction until you got the entry you wanted. Here’s how we can get the third entry in a 5-length list:

The underscores are wildcard characters in Prolog, and they will accept any value and just toss it out; they’re practically just placeholders for values. Overall this looks ugly, right? I agree, and if you’re doing things like this, then that’s normally a code smell that you may not be using Prolog in the best way.

That’s all that we’re gonna cover with lists in Prolog; you can certainly get deeper than this – but there’s still really not much to them. We’ll showcase how you can use them in real-world examples in the next post.

Final thoughts

If I had to place emphasis on understanding one core concept in Prolog (other than basic facts, rules, and queries), it would be unification. No other language has this capability quite like Prolog does – and it’s really, really powerful (and not the easiest thing to understand on the first go through). With the power of unification, we’re able to build complex rules and queries that emphasize recursion, lists, math, and much more. I hope you enjoyed this post; we reviewed a lot – and we’re going to be using every bit of knowledge we gained here in the final post of this series where we’ll build a sudoku-solver with very little logic compared to how an imperative language might approach that problem.

Stay tuned!

Note: You can check out all the examples in this post in my Prolog Demo GitHub repo!

Declarative Programming with Prolog – Part 1: Getting Started

Have you ever run into a programming problem that seemed difficult to solve, but you could easily describe the rules of the problem? Think about games like tic-tac-toe, checkers, and sudoku; as humans, we’re able to understand the rules of these games pretty easily – and sure, you could write a program to implement them, but it wouldn’t be nearly as easy as just teaching a friend how they work. But why does this even matter? We know programming is different than just talking to another person, so why even try to compare them? Well, what if we could just write a program where we instead describe the rules that we need to abide by instead of explicitly telling our program how to do everything? I know that sounds crazy – but it’s 100% doable, and there are tools out there like Prolog that are meant just for this.

Prolog is a really awesome language that allows you to do things that many other languages such as C, JavaScript, Python, C#, Ruby, etc. just can’t do. That’s because Prolog isn’t like these other languages; it’s a declarative language, whereas those others (and countless more) are imperative. We’ll come back to what those terms mean here in a bit.

Prolog isn’t a new language, either; it’s actually been around for over 45 years at this point – so is there a point to even learning it today? You bet there is – because even if you don’t end up using it in your career, it really makes you think differently about how you can solve problems when programming, and that’s always a good thing. That’s not to say that you won’t use it in your career, however; Prolog is heavily used in the natural language processing field, and here’s a fun fact: a large portion of IBM’s Watson is written in Prolog.

This post is the first part of a three-part series over Prolog, and today we’ll dig into how Prolog actually works and just barely get our feet wet with some code over its core concepts. In the next couple of posts, we’ll dive deeper into Prolog and show off how its declarative architecture can help you solve problems that imperative languages would really struggle with.

Declarative vs Imperative Programming

Before we talk more about Prolog, we need to go over what makes it so different. Most programming languages are imperative, and that means that you interact with them by giving a compiler or interpreter an exact set of instructions to execute – and that’s your code. Your program will only do exactly what you code it to do; if you’re like me, this probably sounds completely normal, right? Could things be any other way? If you don’t code exactly what you want to happen – then what else would your program do? This is where declarative programming differs from imperative programming. With declarative languages, you just describe what you want your code to do, and not how to do it. Declarative programming is all about building a control flow that involves data and relationships, and then letting the language’s implementation take over from there.

This probably sounds ridiculous if it’s your first time hearing about this paradigm – but there’s actually a good chance you’ve already experienced it before. Ever heard of SQL? That’s a prime example of a declarative language – and easily the most common one in use today. Think about it. When you issue a SQL command, you don’t tell the server how it should query/create/delete data; you just tell it what to do, and it does it. You don’t manually build a loop to modify multiple records, or explicitly code out how to filter columns. SQL just does it – and it wouldn’t be nearly as powerful if it didn’t function this way.

Prolog behaves similar to SQL. You build what’s called a knowledge base using facts and rules, and then you query that knowledge base. In fact, those are the three different concepts that make up everything you can do in Prolog: facts, rules, and queries. Sounds simple, right? Honestly, it really is – and I’m excited to show you how powerful this language can be.

Let’s get started!

Our First Script

Let’s start off with a simple example.

Those first 3 lines are facts, and through them we’re just defining some simple data points. The word like in the fact is called a predicate, and it can be anything we want it to be since we’re defining it (instead of querying it). The arguments are still called arguments, just like we’re familiar with, and since there are 2 arguments, this predicate has an arity of 2.

That last line in this script is a rule, and rules allow you to add in logic such as querying facts and other rules. A rule is denoted by the syntax ” :- ” that you see above (which translates to “is true if”), and everything that comes after that operand is called a subgoal. The subgoals are separated by commas, and each of them must return true for the entire rule to return true. If this is confusing, it may help to see how we would read the entire friend rule in English:

This sounds cool, right? We don’t have to specify what the two people like in common – Prolog will figure that out for us. Additionally, if they don’t have anything in common, then Prolog will figure that out too. Let’s run this script and issue some queries.

Note

One thing needs to be said here. While both facts and rules may look similar to functions in other languages – they’re very different. Prolog doesn’t even have functions – it just has facts and rules that handle all the logic. Unlike functions, facts and rules can only respond with true or false.

Running Our Script

There are plenty of implementations of Prolog out there, but the most feature-rich implementation is SWI-prolog. That’s what we’ll use to run our scripts.

First, we query two of our facts. No surprise there, the first query returns true because Tim likes soccer, and the second one returns false because John doesn’t. The next two lines are where we query our friend rule, and we see that Tim and Gretchen are friends because they have some similar like – but not Tim and John. It might not seem like much is happening here – but think about it. We never had to tell Prolog what to look for when finding matching values among our facts – it just did it! There’s no way we could do this in an imperative language without explicitly telling our program to manually loop over a set of certain set of criteria. It seems like magic, and if you’re not impressed yet – then don’t worry, we have a lot more to discuss in this series.

A Slightly Deeper Example

Let’s take the concepts we’ve already discussed and take them to the next level. Check out example-2.pl below:

A lot of this should look familiar – the only real difference is that we now have two different fact predicates, and our rule looks a little different. The food_flavor rule combines two facts, and works as follows:

And we can query it like this:

We defined that velveeta has a food type of cheese, and in a later fact we stated that cheese has a flavor of savory – so the first query returns true because all of the subgoals were met. Meanwhile, taco_bell doesn’t match up with the flavor of savory, and we see how that returns false. If you didn’t before, you can probably start to see the power of how prolog handles data – but I mentioned that we were gonna take this a step further, so let’s learn about variables.

Variables

In prolog, all lowercase arguments are called atoms – or just constants; uppercase arguments, on the other hand, are variables. Whenever you use variables, Prolog attempts to match them all with a value (and it’s entirely possible that a variable could match to multiple values), and it does this through a concept called unification (we’ll get to that more in the next post in this series). We’ve been using variables and unification all along in our rules; for example, when we queried the food_flavor rule and passed in the atoms taco_bell and sweet, Prolog unified them to the variables X and Y. It then proceeded to unify the Z variable to some other value to determine if the rule succeeded or not; we just never saw that because we never asked for Z!

We can query for variables too though, and that’s what I want to show here. Let’s say that we want to query our flavor facts to find all food types that are savory:

Prolog unified the variable What to two possible values: tacos and cheese. We can take this a step further and pass a variable in to our rule. Let’s see what happens when we query for all food brands that are sweet:

Again – we get two results. You might think it’s weird that we’re passing in a variable that’s just getting unified to another variable in the rule – but Prolog doesn’t care and ends up handling it beautifully. We could even pass in variables as both arguments to get the cartesian product of all successful matches!

That’s it?

No, that’s certainly not it – but I didn’t want to scare you off too fast. Prolog is different, that’s for sure, but it’s really powerful – and I hope you’re beginning to see that. In the next post, we’ll dive deeper into core Prolog concepts such as recursion, lists, and more into unification – all to prep for the third and final post in this series where we showcase how you can really use Prolog for some cool real-world stuff.

Stay tuned for the next post!

Note: You can check out all the examples in this post in my Prolog Demo GitHub repo!


References

Many of the examples here were based on (but modified) from the book Seven Languages in Seven Weeks. This book is what gave me the inspiration to write this blog series, and I highly recommend it if you want an overview of not only Prolog, but several other languages as well.

Seven Languages in Seven Weeks Book Cover

Running a Rails App with Docker

Docker is quickly becoming the new cool-kid tool around town, and despite the fact that it’s still in rapid development, it has become stable enough over the past couple of years to the point where you can actually use it for some of your production apps. If you Google around, you’re bound to find plenty of tutorials that review how to get started with Docker and (insert your favorite language) – but I want to take it a step further. I don’t want to just show you how to run a Rails app with Docker; I want to build with you an actual use-case scenario where using Docker makes a ton of sense. See, where Docker truly excels is when you have multiple services that are all communicating together. These services can be practically anything: a web server, app server, database, background job processor, etc. – and when you start having a bunch of services all relying on each other, it makes development more difficult, especially collaborative development. What if my computer’s running a different version of one of these services than yours is? That could easily cause inconsistencies if we’re developing in a team environment. In order to eliminate these kinds of issues, incorporating something like Docker really becomes appealing.

To take things a step further, we’ll also be talking about docker-compose – a wonderful tool by the Docker team that helps manage the running of multiple containers.

Let’s talk a bit about what we actually want to build.

The Scenario

ERD over Rails Docker Blog demo

We want to build a simple Rails blog API with userpost, and comment resources; you can see the ERD of our planned database over to the right. Whenever we create a comment (i.e. make a successful POST request to /comments), we want to send an email to the creator of the post, letting them know a comment was created. However – we don’t want to send this email synchronously; we want to offload it to a background job processor to handle the actual sending of the email. Last but not least, instead of actually sending this email to a real person while in development, we just want to capture the email so that we can inspect and debug it. To do this, we’ll use a really cool tool called maildev which sets up an SMTP server to capture emails, as well as an HTTP server to allow us to view the emails.

This may sound like a lot going on – but don’t overthink it; we’re just sending an email when a comment is created. It just so happens that we have a few more services needed in order to make this happen – which is why this is a perfect scenario to showcase how powerful Docker is.

What is Docker?

Just to make sure we’re all on the same page, let’s briefly review what Docker actually is. Docker is a platform for running and managing what are called containers – i.e. lightweight pieces of software that are geared to running a single specific process. They’re sort of like virtual machines, but much, much smaller and they each have a specific job to do. Containers are also designed to be spun up and down very quickly, which helps make them appealing. In our above example, we would have separate services running for the Rails app, background job processor, maildev – and more.

Getting Started

Note: For the following code, we’ll be using Rails v5.0.2.

We’re ready to start building our app – but for this first segment, we’re not even gonna use Docker. There’s a little bit of setup for us to get through before we take that step.

Run the following command line commands to get our base database structure going (for a real app, we would use a serious database such as PostgreSQL, MySQL, etc.; for now, we’ll stick with the default SQLite).

Before we migrate our database, let’s add some simple seeds – just to get some dummy data in there:

Good, now we can migrate our database and run our seeds:

Once we have all of that set up, let’s go ahead and add the sidekiq gem to our Gemfile, because that’s what we’ll be using as our background job processor.

Sidekiq depends on Redis – which means we’ll be tying that into our setup later on as well!

Now install the gem:

We now have our app set up, so let’s go ahead and generate our mailer:

This will create the file app/mailers/comment_mailer.rb – which we’ll now edit to add in a new_comment mailer function:

As you can see, this mailer will send only to the author of the post that this comment is for. But exactly what does it send? That’s what we need to add next – our HTML and text email templates. First we need to create the templates:

And now we need to add some brief content into each of them:

Perfect, now that we’ve got our mailers set up – we actually need to update our comments_controller.rb to deliver that email (and specifically we need to update the create action):

The only line we need to add is line #9 where we make the CommentMailer call. Also, notice that we’re using the deliver_later method when we haven’t even hooked up a queuing system yet. This function tells ActionMailer to use ActiveJob to send out the emails – and if you don’t have any background job processor in place yet, then ActionMailer will just process the job synchronously. Fun fact.

Go ahead and start your server. As of right now, if you issue a POST request to /comments – such as this one below – then Rails will successfully create a comment and try to send an email.

You should see a successful response from your curl command, as well as Rails logging out its intent to send out an email. No emails will actually send right now because we haven’t hooked up any email settings such as SMTP credentials – but don’t worry, we’ll fix that in a bit. It’s time to start integrating what we currently have with Docker.

Adding in Docker

Note: From this point on, you’ll need to have both the Docker Engine and Compose installed. If you’re on Windows or Mac, you can just install the Docker Toolbox to get both (and more). Otherwise, you’ll need to install them separately.


To implement Docker, we first need to add in a Dockerfile so that we can build our Rails app as a Docker image. Add in the following Dockerfile to the root of your project:

There’s nothing special about this Dockerfile; at the time of this writing, you can find this exact example straight from the Docker site about how to create an image from a Rails app. We could now issue a “docker build” command to create this image – but let’s hold off on that. I mentioned in the intro that we’ll be using docker-compose to both build and run all of our images. To use docker-compose, however, we first need to add in another file called docker-compose.yml to the root of our project:

I won’t lie – it looks like there’s a lot going on here, but stay with me. All we’re doing here is preparing 4 different images (2 of which are built with this project’s Dockerfile) that will all be run at the same time – in the same network. That last statement’s really important, because networks in Docker are really cool. Since each container technically has its own unique address – it’s a little difficult for images to know how to communicate with each other. Docker networks are great because they will automatically resolve the hostname of a container based on what you name it – so that if you want to communicate with a container that’s titled sidekiq in the docker-compose.yml file, then you only have to specify its address by its name – “sidekiq”! You do still need to include any ports that the service is running on, though.

If this all sounds confusing – don’t worry, you’re not alone. This is difficult to get a grasp on the first time you see it (and many times after that, too) – but believe it or not, we’re almost done here, so let’s keep going.

Next, we need to add a few settings in our development.rb config file to set up SMTP to send to the Maildev container’s SMTP server:

The Maildev container’s SMTP server runs on port 25 – which is the default port for SMTP – and you can see here that we’re locating the server just by the string “maildev.” This works because when we run our setup using docker-compose, it creates a network which will resolve that hostname and send the request to the right container. Port 25 is also exposed by default on that container – but only to other containers in the network; you can’t access it outside of the Docker network.

Now, we need to tell ActiveJob (Rails’ built-in background job wrapper) that we want to use the sidekiq adapter to queue up our jobs. This officially throws Sidekiq into our application:

Finally – there’s one last thing; sidekiq by default assumes that redis is running on localhost:6379 – but since the redis server is running in a different container than our Rails app, we need to change this. We need to instead direct our redis traffic to the actual redis container, which we can easily do by using the hostname “redis.” To do that – we just need to add a simple initializer:

And that’s it – we’re done!

Running our Application

This part’s super easy; we’ve set everything up, and now we just need to run our app with docker-compose!

This will handle building all of the images that are custom, as well as pulling down and installing the other images from Docker Hub. Our app is officially up and running now with Docker, so let’s test it with the same curl command we had before:

If everything’s set up properly, you should get a successful JSON response that includes the comment record you just created. This command still works because we’re mapping port 3000 of our Rails app container to port 3000 on the Docker host (which is our local machine) – so we can communicate with it the same way we did before. After you issue that POST request, jump to http://localhost:1080 to see Maildev in action – and you can see the exact email you just created!

Example of Maildev

 

Final Thoughts

While Docker is definitely a hot topic right now, will it stand the test of time? Who knows – but the concepts behind containerization are here to stay – that much we know for sure, and Docker is really helping to push that movement forward. If you enjoyed this post about Docker and want to check out how companies use containerization in the real world, you should read more into the microservice architecture. Microservices have been around for a while, but with the popularization of Docker (and containers in general), it’s being talked about a lot more as a viable architecture for even small- to mid-size projects.

You’ve now got the knowledge, so when you get a chance, play around with Docker and see if it’s right for your Rails app. The answer could be yes – or it could be no, and either is okay! Docker is a neat tool, but only you can decide if it fits your project’s needs.

P.S. If you’d like to pull down the code we discussed here, check out the demo based on this blog post.