Build Haskell web services with Servant - Introduction

I’m beginning a series of blog posts motivated by a scream of HEEELP! I sensed in a Reddit post (ok maybe I’m exaggerating… was more like a whisper), the OP was pretty much complaining about the lack of resources to build real stuff with Servant, the community just pointed him to the “amazing docs” and the Servant paper to learn about it, others just discouraged it as “Servant is not for beginners”. Well I don’t think I share that stand, this should not be the way to learn stuff as a beginner, maybe for academics it is, but for practitioners coming to Haskell from mainstream-land (JS, Python, TS, Go, etc) it is basically given that lots of resources should exists out there to learn or implement something, this then, is my contribution to that cause.

What we will build here is just a Servant web service that manages posts of a blog application, our focus will be in Servant unless we derail into other topics that will allow to increase further this series of posts.

Setting up the project

We will begin by setting up a Haskell project using stack. What’s that? Why “stack” I hear you say? Well this is a beginners introduction so the less we have to mess up with tooling the better, if you know your way around Nix or cabal you can use them, but I’ll doubt you’ll need these sort of guides 🤔; whatever, let’s begin by creating an empty project, I’ll call this blog-ws:

$ stack new blog-ws

Now, this will create a project structure that for the most part is ok, however I’m going to delete the package.yaml file, this is because I don’t want to use hpack to manage the .cabal file, you can leave it there if you want just remember in that case to add your dependencies to the package.yaml instead of the blog-ws.cabal file.

hpack is a tool that was developed to be able to declare your project as a .yaml file and fix some short-comings of .cabal files, when stack sees a package.yaml file in the project then it will use hpack to auto-generate the .cabal file from it. Many like this and many don’t, it comes down to personal preference, but in case you want my advice: in the end you’ll still need to get familiar with editing .cabal files for that time when builds get tricky, so why not just remove that extra layer that hpack is and get your hands dirty from the beginning?

.
├── ChangeLog.md
├── LICENSE
├── README.md
├── Setup.hs
├── app
│   └── Main.hs
├── blog-ws.cabal
├── src
│   └── Lib.hs
├── stack.yaml
└── test
    └── Spec.hs

Now let’s open up the blog-ws.cabal file (or the package.yaml if you didn’t remove it) and list some dependencies we will need under the library > build-depends section:

library
  build-depends:
      base >=4.7 && <5
    , mtl
    , text
    , aeson
    , say
    , time
    , servant-server
    , servant-errors

Let’s begin with those and will add more as we need them. Now let’s build it, but do it fast:

$ stack build --fast

Ok, if this was your first time using stack that will anything but fast. This is because a resolver will need to be set up on your computer, that will download a compiler and the dependencies that are assured to work with that compiler version. In any case, we do this one-off build just to trigger stack to download the dependencies we’ve listed.

Introducing the domain types

Let’s add a new module Blog.Posts.Types that will contain our domain types, we’ll need a Post type to model blog posts and we’ll also introduce some additional types that will become handy for creating or updating posts via API later. Create the following file under src/Blog/Posts/Types.hs:

module Blog.Posts.Types where

import Data.Text (Text)
import Data.Time (UTCTime)

data Post = Post
  { postId          :: Int
  , postTitle       :: Text
  , postContent     :: Text
  , postAuthor      :: Text
  , postPublishedAt :: UTCTime
  } deriving (Eq, Show)

data NewPostDto = NewPostDto
  { npdTitle   :: Text
  , npdContent :: Text
  , npdAuthor  :: Text
  }

data EditPostDto = EditPostDto
  { epdTitle   :: Maybe Text
  , epdContent :: Maybe Text
  , epdAuthor  :: Maybe Text
  }

Why to use these extra Dto types? Well, its baggage from my past in the OOP world, a Data Transfer Object is a practice to specify how data should be transferred, I know here we specify no behavior that’s not important, in fact maybe the name is not important at all, the key thing is that when we allow a Post to be created via our API, we don’t want/need the client to send the publication date (assuming posts can’t be drafts and once they’re sent they are published), nor client should send a post ID in the body request to create a Post, you see Haskell is static and as such has trouble with dynamic fields on records. For this same reason, we use different DTO types for creating or updating a post. Maybe for example, will allow us to cope with the fact that when updating a Post the client might only want to update the title and leave other info unchanged, so we use it to handle the absence of those fields in the request body.

Continuing with the setup, we need to add this new module in our blog-ws.cabal file, under library > other-modules section:

library
  other-modules:
      Paths_blog_ws
    , Blog.Posts.Types

Now, let’s set up ghcid, this is an amazing tool that just watches for source file changes and reloads the code to be type checked, it can also run the app or the tests, but we will see how to do that later, for now, if you don’t have it installed head to the GitHub repo to see how to install it, probably stack install ghcid will do it. Once its installed just let it running in your terminal:

$ ghcid

And after the project loads, you should see something like:

All good (3 modules, at 12:56:22)

To test it is working fine, let’s do some refactor, take a look at our DTO types:

data NewPostDto = NewPostDto
  { npdTitle   :: Text
  , npdContent :: Text
  , npdAuthor  :: Text
  }

data EditPostDto = EditPostDto
  { epdTitle   :: Maybe Text
  , epdContent :: Maybe Text
  , epdAuthor  :: Maybe Text
  }

You may see these types have similar structure and indeed we can avoid the repetition there. They share the same fields, the only thing different is that when creating a Post we will require all of the fields (title, content, author), whereas while updating the Post we will allow the client to only update those fields he wants to change, maybe only the title will change, for this reason fields use Maybe to signal they might be there or they might not. However we can pull out the context, we will need the Identity functor (actually just a type here) like this:

data PostDto f = PostDto
  { pdTitle   :: f Text
  , pdContent :: f Text
  , pdAuthor  :: f Text
  }

type NewPostDto = PostDto Identity
type EditPostDto = PostDto Maybe

We just introduce a “generic” PostDto that will be used for both adding new post as well as editing an existent post, it will need to take the context as a type variable (f), and we change both NewPostDto and EditPostDto to be a type aliases instead, just as helpers. Save the file and instantly ghcid will complain:

/workspace/haskell/blog-ws/src/Blog/Posts/Types.hs:22:27-34: error:
    Not in scope: type constructor or class ‘Identity’
   |
23 | type NewPostDto = PostDto Identity
   |                           ^^^^^^^^

Great, very fast development feedback loop there, let’s fix the error by importing the Identity type at the top of the module:

import Data.Functor.Identity (Identity)

Save again and… All good!

A Servant preamble

I know I know, wasn’t this a Servant tutorial? we haven’t even seen it yet! Let’s beginning by pointing out some things about Servant.

Some people in the community don’t recommend learning Servant to beginners because it exposes us to advanced features of Haskell, but I don’t think I agree, while it is true that uses advanced Haskell trickery (basically type-level programming) if you are learning Haskell and are interested in Servant then there should be learning materials out there that takes you by the hand and helps you to learn it, you should be the one to judge whether something is/feels advanced or not. So these series are my attempt to provide those Servant learning resources for the ones that don’t enjoy digging through source code, hackage, old GitHub issues and reddit posts to learn how to build stuff with it. Let’s continue!

Servant allow use to define a web service using a type level DSL, all that this means is that we can “describe” the interface of a web service in pure types, that is, what are the endpoints?, what is the payload each endpoint expects?, what does the endpoint responds with?, what kind of authentication requires?, etc. This gives us the benefit of ensuring the web service is well defined at compile time and the ability to use types to generate other interesting stuff such as documentation or http clients for free.

If you have not tinkered with type level stuff in Haskell, fear not, treat Servant DSL as a meta-language and nothing more, the only downside for beginners is that some times type errors can become hard to understand but not by much if we’re logical about it. As mentioned earlier, our API will manage blog posts, so let’s describe a simple CRUD web service to manage them (maybe this is a headless CMS or something along the lines, that’s not important).

To serve a Servant web service all we need is to build a WAI Application, that latter can be fed to the run function from WARP package.

What’s that? what is WAI I hear you say? Ok, think about python’s ASGI/WSGI but for Haskell land, WAI package provide types that are shared among many web frameworks or libraries so that the framework it self doesn’t need to care about how to serve the web application, parse requests or build responses. This in turn is used by WARP to serve the application, in this sense WARP is just a web server that has support to serve web applications that conform to WAI.

Shall we continue?…

For constructing our Application using Servant we need these 3 things:

1. The type of the API (the description of the web service itself).
2. A “server” which is composed of the handlers we want to evaluate when requests arrive to the HTTP endpoints.
3. Call a serve function (or any of its variants) providing both the type of the API (1) and the server (2).

Pretty easy! nothing advanced here!

Constructing the API Type

This is were the Servant type level DSL comes in, we will use the “primitives” provided by the servant package to describe our own API, there are two main pieces we will need:

1. :<|> (aka “The combinator”) think of it like an overly complicated + sign, its only role is to combine multiple routes or endpoints together, for example, our CRUD will require at least 4 different endpoints, but all of them together should be part of one single API, so we will describe our endpoints “separately” and combine them into a single type using this type level operator.

2. :> (aka “The URI path separator we often see in URLs”) that is, we can think of this of being equivalent to /, for example GET /blog/posts will be roughly translated as "blog" :> "posts" :> Get ..., note the allowed method comes last.

Now, I recommend looking at the Servant docs to see what “pieces” are available and how they should be used, just remember that in order to use them we need to enable the TypeOperators language extension. Let’s say we want to create the API with the following endpoints:

This is a simple CRUD over a REST-like API for managing blog posts, nothing special really. Now, let’s see how to “express” the first endpoint (GET /blog/posts) using the Servant DSL:

Ok let’s explain what '[JSON] and [Post] mean here.

When describing endpoints of a web service we need to specify what is the allowed Content-Type of the request, that is, the format of the request body, here we’re saying “hey! this endpoint deals with JSON", thus we expect the request to have the Content-Type: application/json header, and also we’re replying with JSON, theoretically we could work with several content types, maybe we also support plain text (text/plain) and form data (application/x-www-form-urlencoded) in that case we can add those to the list of content types like this '[JSON, PlainText, FormUrlEncoded]. Notice that this list is prefixed by a tick mark ', this is because endpoints are types, we’re kind of programming at the type level, and using a simple list [] won’t work, so '[] is for type level list, for this to work we’ll need to enable the DataKinds language extension.

Now, [Post] is the endpoint’s return type, more specifically, is the type of the data contained in the body of the response returned by the endpoint, in our case we want to be able to get all posts when querying this endpoint, so it makes sense for this endpoint to return a list of posts and thus the type [Post] is used. Maybe you’re wondering, “why is this list not prefixed with '? (as it was for content types)", well, in order to answer the question this would have to stop being a beginner’s introduction to Servant, so you’ll get an overly simplified (not totally accurate) answer. While using a type level list (prefixed with ') what conceptually means is that we are referring to a list or constructing one in place (at the type level) as if we were using it as a parameter to some type level function, whereas using [Post] without ' conceptually means “this is a type”, i.e. not constructing nor declaring a list, but just stating the fact that “this is the type of expressions returned by the endpoint”. If you don’t get it don’t worry, just read it like “I want to return a list of Posts” rather than “I want to construct a type level list that contain the type Post in it”, in that sense we can think of Get as being a function at the type level that as its first arguments expects a list of types, and as the second just a single type.

The Get there in case you’re wondering is just the method allowed by the endpoint, use Get for allowing GET requests, Post for POST requests, Put for PUT, etc.

Now, exercise time! can you translate all other endpoints by you self? would you like to try? No? oh I see “just chilling” right?, let’s continue then.

Ok, we’re trying to make this posts web service into a RESTful web service, pedantics will notice we’re missing to specify some headers too, for example a Location header should be in the response when creating a resource, but let’s not bother with that for now to keep things simple. We just need to combine these endpoints into a web service, and remember we can use the ugly combine operator :<|> (nice exercise for the fingers), so basically do endpoint1 + endpoint2 + endpoint3 + ... like this:

"blog" :> "posts" :> Get '[JSON] [Post]
   :<|> "blog" :> "posts" :> Capture "postId" Int :> Get '[JSON] Post
   :<|> "blog" :> "posts" :> ReqBody '[JSON] NewPostDto :> PostCreated '[JSON] Post
   :<|> "blog" :> "posts" :> Capture "postId" Int :> ReqBody '[JSON] EditPostDto :> Put '[JSON] Post
   :<|> "blog" :> "posts" :> Capture "postId" Int :> DeleteNoContent

Ahh… look at that 😌, who said only Rust can have ugly messy types? 😏

Now, that long line of code there is just a single type, it is the type of our web service in the same vein as Bool is the type of True, or String the type of "Hola", so maybe we need to introduce a type alias so that it is more tractable to refer to it, let’s create a new module Blog.Posts.Api and put the alias there:

-- File: src/Blog/Posts/Api.hs
module Blog.Posts.Api where

type BlogService
  = "blog" :> "posts" :> Get '[JSON] [Post]
  :<|> "blog" :> "posts" :> Capture "postId" Int :> Get '[JSON] Post
  :<|> "blog" :> "posts" :> ReqBody '[JSON] NewPostDto :> PostCreated '[JSON] Post
  :<|> "blog" :> "posts" :> Capture "postId" Int :> ReqBody '[JSON] EditPostDto :> Put '[JSON] Post
  :<|> "blog" :> "posts" :> Capture "postId" Int :> DeleteNoContent

I may have some weird tastes for formatting code, but hey, this is my blog 💅. And also let’s take into account that every endpoint is just a type, we’re making types from types by composing them, so Rails/Laravel enthusiasts (welcome) are free to do this as well:

type IndexPost = "blog" :> "posts" :> Get '[JSON] [Post]
type ShowPost = "blog" :> "posts" :> Capture "postId" Int :> Get '[JSON] Post
type CreatePost = "blog" :> "posts" :> ReqBody '[JSON] NewPostDto :> PostCreated '[JSON] Post
type UpdatePost = "blog" :> "posts" :> Capture "postId" Int :> ReqBody '[JSON] EditPostDto :> Put '[JSON] Post
type DestroyPost = "blog" :> "posts" :> Capture "postId" Int :> DeleteNoContent

type PostsAPI = IndexPost :<|> ShowPost :<|> CreatePost :<|> UpdatePost :<|> DestroyPost

Now, you may be complaining about repetition of "blog" and "post" on each endpoint, well these are just types right? so we can pull them out:

type IndexPost
  = Get '[JSON] [Post]

type ShowPost
  = Capture "postId" Int
  :> Get '[JSON] Post

type CreatePost
  = ReqBody '[JSON] NewPostDto
  :> PostCreated '[JSON] Post

type UpdatePost
  = Capture "postId" Int
  :> ReqBody '[JSON] EditPostDto
  :> Put '[JSON] Post

type DestroyPost
  = Capture "postId" Int
  :> DeleteNoContent


type PostsAPI =
  "posts" :> (IndexPost :<|> ShowPost :<|> CreatePost :<|> UpdatePost :<|> DestroyPost)


type BlogService = "blog" :> PostsAPI

Ahh, looking better 😌.

And, let’s not forget that for all of this to work we need to both import our domain types and the Servant package to have access to the types and type level operators we’re using, also we need to add those language extension we mentioned before so that we can use the type-level trickery that is going on here, so this is the whole src/Blog/Posts/Api.hs file:

{-# LANGUAGE DataKinds     #-}
{-# LANGUAGE TypeOperators #-}

module Blog.Posts.Api where

import Servant hiding (Post)
import Blog.Posts.Types (Post, NewPostDto, EditPostDto)

type IndexPost
  = Get '[JSON] [Post]

type ShowPost
  = Capture "postId" Int
  :> Get '[JSON] Post

type CreatePost
  = ReqBody '[JSON] NewPostDto
  :> PostCreated '[JSON] Post

type UpdatePost
  = Capture "postId" Int
  :> ReqBody '[JSON] EditPostDto
  :> Put '[JSON] Post

type DestroyPost
  = Capture "postId" Int
  :> DeleteNoContent


type PostsAPI =
  "posts" :> (IndexPost :<|> ShowPost :<|> CreatePost :<|> UpdatePost :<|> DestroyPost)


type BlogService = "blog" :> PostsAPI

Yes, pure types there! Note that we need hiding (Post) while importing Servant to avoid clashing with our own Post type.

Finally, if you haven’t do so already, let’s add the module to the blog-ws.cabal file:

library
  other-modules:
      Lib
    , Blog.Posts.Types
    , Blog.Posts.Api

Conclusion

This is a good point to end this introduction post, we got our project setup, created our domain types and most important of all, defined the type for our web service using the Servant DSL, recall that for any Servant application we need three things:

1. The type of the web service API.
2. A “server” composed of the handlers we want to evaluate when requests arrive to the HTTP endpoints of the web service.
3. Use one of the serve flavored functions providing both the type of the API (1) and the server (2).

So 1 we already have it, next post we will tackle 2 and 3! so stay tuned 🎶.