Creating web sites with Suave How to contribute to F# Snippets
The core of many web sites and web APIs is very simple. Given an HTTP request,
produce a HTTP response. In F#, we can represent this as a function with type
Request -> Response
. To make our server scalable, we should make the function
asynchronous to avoid unnecessary blocking of threads. In F#, this can be
captured as Request -> Async<Response>
. Sounds pretty simple, right? So why
are there so many evil frameworks
that make simple web programming difficult?
Fortunately, there is a nice F# library called Suave.io that is based exactly on the above idea:
Suave is a simple web development F# library providing a lightweight web server and a set of combinators to manipulate route flow and task composition.
I recently decided to start a new version of the F# Snippets web site and I wanted to keep the implementation functional, simple, cross-platform and easy to contrbute to. I wrote a first prototype of the implementation using Suave and already received a few contributions via pull requests! In this blog post, I'll share a few interesting aspects of the implementation and I'll give you some good pointers where you can learn more about Suave. There is no excuse for not contributing to F# Snippets v2 after reading this blog post!
Getting started with Suave
I recently did a couple of talks about Suave at user groups and conferences and many of them have been recorded. There are also a couple of nice examples online and some good documentation on the official web site. So if you want to learn more about Suave, here are some links for you:
-
Channel 9 Interview. Seth Juarez did an interview with me when I was in Redmond and I did a quick 20 minute demo showing how to Deploy an F# Web Application with Suave to Azure. This is the last part of a mini series, so you might also want to check out Making the Case for using F# if you are new to F#, Domain Modeling in F# and Type Providers in F#.
-
NDC Oslo Talk. Next, I talked about Suave at NDC in Oslo. The talk shows two demo application - a web portal showing weather and news and a simple chat written using agents. The talk End-to-end Functional Web Development has been recorded and you can also get the full source code. and slides. It is also worth noting that I'm using the awesome F# Atom plugin in the talk, together with some custom FAKE build scripts to get a nice live reloading when developing the web sites.
-
Community for F# Talk. Henrik Feldt who is one of the Suave contributors did a nice talk Suave from Scratch on the Community for F# channel. This shows many more Suave features and so it is a great follow-up to the above. Also, Henrik is showing Suave on Mac using Xamarin Studio, so you can see that it truly is cross-platform.
-
Web Site and Dojo. For more information, there is a bunch of examples and documentation on the official Suave web site. This includes various ways of deploying Suave applications too. If you then want to get some hands-on experience, try completing the simple Suave Dojo that I put together!
Introducing F# Snippets v2
As already mentioned, I started using Suave for the new version of the F# Snippets web site. The web site is basically a pastebin for F# code snippets. The nice thing is that it uses F# Formatting for formatting the code snippets and generating tool tips. I never released the source code for the old version, because it was simoply too ugly. The new version fixes this!
- The source code is on GitHub - to run it locally, you'll need to download sample data as discussed in the README.
-
The prototype runs on Azure - this is
automatically deployed from the
master
branch in the GitHub project and it runs as Azure Website. - And here is a list of remaining issues before it can replace the old version - the project is quite simple, so this is a great place where you can contribute!
The previous version of F# Snippets stored all data in an SQL database. When creating the new one, I was wondering what is the best option given the size of the web site. It turns out that the meta-data about all the snippets is small enough to fit in memory (about 1MB in JSON format) and so the new version is a lot simpler.
It keeps the meta-data in memory. The formatted snippets are stored in local file system (when testing things locally) or in Azure blob storage (when running on Azure) - though you can also use Azure storage during development. When the meta-data change, it is also saved to a JSON file in the blob storage (so that it can be reloaded if the application is shut down).
You can find more details in the project architecture section of the project README document.
Interesting Suave snippets
There is a number of things that make Suave really nice to use. As you can have a look at the materials above to learn everything about it, I want to give you just a few examples based on my experience with F# Snippets.
The first nice thing about Suave is that it is a library rather than a framework. This
means that you are in control of starting and running the server. This makes it easy
to deploy it to Azure, Heroku or anywhere else. In F# Snippets, we have one entry-point
in the app.fsx
file. This composes the server from individual components.
Composing server from web parts
The following code snippet shows how the server is composed. As you can see, we have functionality for showing the home page, displaying snippets, inserting new snippets, listing snippets and the RSS feed:
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30: |
|
The choose
combinator takes a list of web parts and composes them. A Suave web
part is essentially one of those functions from the introduction - web parts can handle
requests and produce response. Here, we are building a single web part that goes through
the web parts in the list and uses the first one that can handle an incoming request.
The path
combinator is used to restrict what requests a web part handles - so for
example path "/" >>= Home.showHome
means that we should display the home page if the
request is for the path /
. A very nice function is pathScan
- it takes an F#
format string and builds a web part that recognizes requests to URL with the specified
pattern. We can, for example, say pathScan "/raw/%s/%d"
to detect URLs such as
/raw/cJ/5
.
Displaying snippets with DotLiquid
The Suave library does not force you to use any specific templating engine and I actually
used Suave for some time with just string concatenation or str.Replace
. But if you want
to use some templating library, it is really easy to add support for it. To see just how
easy, look at my pull request adding support for DotLiquid.
We're using DotLiquid in F# snippets, so here is how the
code looks.
The code sample below
shows how we handle request to display a snippet. We get the snippet ID, get information about
it from the meta-data and read the file from storage. If everything succeeds, we create
a record FormattedSnippet
and pass it to the template loaded from snippet.html
:
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: |
|
You can find the full template on GitHub.
The value of the record is exposed as model
and we can access its properties in the
template. For example, the heading is generated by <h1>{{ model.Details.Title }}</h1>
.
Checking F# code during insertion
The new F# Snippets web site reports all the errors in your F# code on the fly when you are inserting the snippet. Go to the insert snippet page, type some invalid F#, wait a second and you should see the compiler errors and warnings!
The implementation of this uses a simple JavaScript with timer and it calls the /insert/check
API end-point implemented by the server. This then returns a simple JSON with a list of
the errors and warning.
This is another elegant piece of F# code that uses Suave composable web parts and the JSON type provider from F# Data to generate the JSON response. Check out the following snippet:
1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: |
|
There are a few nice things worth mentioning:
-
The example shows an interesting use of the JSON type provider. We give it a sample JSON (list with one error), but we're not using it to read data but instead to generate response. As you can ee on line 20, we can then use the provided type
Errors.Root
to easily build a JSON value representing the error or warning. -
We need to disable all caching in the HTTP response. To do this, we use composition of web parts. We define
noCache
which sets all the different HTTP headers required for this (lines 6-9) and then we use it when producing the result on line 24.
The compositional nature of Suave means that you can really easily define reusable components and structure your code in the way that works for you. For F# Snippets, I wanted to make the project easy to contribute to, and so there is a fairly large number of small independent files implementing the different components.
Summary
This blog post had two purposes. First, I wanted to share some of the resources that you might find useful if you want to learn about web development with F# using Suave. There are many more information available on the internet, including blog post from Scott Hanselman and a cool series by Claus Sørensen, so my list is just scratching the surface!
My second secret goal was to convince you to contribute to the new F# Snippets project. Writing the prototype was a lot of fun and I think you'd have fun contributing too. There is also a very large number of features that people asked about (commenting, search, clustering, suggesting tags, etc.), so I think anyone will find something interesting. To start with, there are a few high-priority issues that need to be resolved before we can replace the old version.
from Suave
from Suave
from Suave.Http
from Suave.Http
from Suave.Http
Full name: Fssnip-suave.app
Full name: Suave.Http.choose
Full name: Suave.Http.Applicatives.path
Full name: Microsoft.FSharp.Core.Operators.id
Full name: Suave.Http.Applicatives.pathScan
{Html: string;
Details: obj;
Revision: int;}
Full name: Fssnip-suave.FormattedSnippet
val string : value:'T -> string
Full name: Microsoft.FSharp.Core.Operators.string
--------------------
type string = System.String
Full name: Microsoft.FSharp.Core.string
val int : value:'T -> int (requires member op_Explicit)
Full name: Microsoft.FSharp.Core.Operators.int
--------------------
type int = int32
Full name: Microsoft.FSharp.Core.int
--------------------
type int<'Measure> = int
Full name: Microsoft.FSharp.Core.int<_>
Full name: Fssnip-suave.showSnippet
from Microsoft.FSharp.Collections
Full name: Microsoft.FSharp.Collections.Seq.tryFind
Full name: Microsoft.FSharp.Collections.Seq.find
namespace FSharp
--------------------
namespace Microsoft.FSharp
namespace FSharp.Data
--------------------
namespace Microsoft.FSharp.Data
Full name: Fssnip-suave.Errors
Full name: FSharp.Data.JsonProvider
<summary>Typed representation of a JSON document</summary>
<param name='Sample'>Location of a JSON sample file or a string containing a sample JSON document</param>
<param name='SampleList'>If true, sample should be a list of individual samples for the inference.</param>
<param name='Culture'>The culture used for parsing numbers and dates.</param>
<param name='ResolutionFolder'>A directory that is used when resolving relative file references (at design time and in hosted execution)</param>
Full name: Fssnip-suave.noCache
Full name: Suave.Http.Writers.setHeader
Full name: Suave.Http.Writers.setMimeType
Full name: Fssnip-suave.checkSnippet
Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.async
type StreamReader =
inherit TextReader
new : stream:Stream -> StreamReader + 9 overloads
member BaseStream : Stream
member Close : unit -> unit
member CurrentEncoding : Encoding
member DiscardBufferedData : unit -> unit
member EndOfStream : bool
member Peek : unit -> int
member Read : unit -> int + 1 overload
member ReadLine : unit -> string
member ReadToEnd : unit -> string
...
Full name: System.IO.StreamReader
--------------------
StreamReader(stream: Stream) : unit
StreamReader(path: string) : unit
StreamReader(stream: Stream, detectEncodingFromByteOrderMarks: bool) : unit
StreamReader(stream: Stream, encoding: System.Text.Encoding) : unit
StreamReader(path: string, detectEncodingFromByteOrderMarks: bool) : unit
StreamReader(path: string, encoding: System.Text.Encoding) : unit
StreamReader(stream: Stream, encoding: System.Text.Encoding, detectEncodingFromByteOrderMarks: bool) : unit
StreamReader(path: string, encoding: System.Text.Encoding, detectEncodingFromByteOrderMarks: bool) : unit
StreamReader(stream: Stream, encoding: System.Text.Encoding, detectEncodingFromByteOrderMarks: bool, bufferSize: int) : unit
StreamReader(path: string, encoding: System.Text.Encoding, detectEncodingFromByteOrderMarks: bool, bufferSize: int) : unit
type MemoryStream =
inherit Stream
new : unit -> MemoryStream + 6 overloads
member CanRead : bool
member CanSeek : bool
member CanWrite : bool
member Capacity : int with get, set
member Flush : unit -> unit
member GetBuffer : unit -> byte[]
member Length : int64
member Position : int64 with get, set
member Read : buffer:byte[] * offset:int * count:int -> int
...
Full name: System.IO.MemoryStream
--------------------
MemoryStream() : unit
MemoryStream(capacity: int) : unit
MemoryStream(buffer: byte []) : unit
MemoryStream(buffer: byte [], writable: bool) : unit
MemoryStream(buffer: byte [], index: int, count: int) : unit
MemoryStream(buffer: byte [], index: int, count: int, writable: bool) : unit
MemoryStream(buffer: byte [], index: int, count: int, writable: bool, publiclyVisible: bool) : unit
| String of string
| Number of decimal
| Float of float
| Record of properties: (string * JsonValue) []
| Array of elements: JsonValue []
| Boolean of bool
| Null
member Request : uri:string * ?httpMethod:string * ?headers:seq<string * string> -> HttpResponse
member RequestAsync : uri:string * ?httpMethod:string * ?headers:seq<string * string> -> Async<HttpResponse>
override ToString : unit -> string
member ToString : saveOptions:JsonSaveOptions -> string
member WriteTo : w:TextWriter * saveOptions:JsonSaveOptions -> unit
static member AsyncLoad : uri:string * ?cultureInfo:CultureInfo -> Async<JsonValue>
static member private JsonStringEncodeTo : w:TextWriter -> value:string -> unit
static member Load : uri:string * ?cultureInfo:CultureInfo -> JsonValue
static member Load : reader:TextReader * ?cultureInfo:CultureInfo -> JsonValue
static member Load : stream:Stream * ?cultureInfo:CultureInfo -> JsonValue
static member Parse : text:string * ?cultureInfo:CultureInfo -> JsonValue
static member ParseMultiple : text:string * ?cultureInfo:CultureInfo -> seq<JsonValue>
static member ParseSample : text:string * ?cultureInfo:CultureInfo -> JsonValue
Full name: FSharp.Data.JsonValue
from Suave.Http
Full name: Suave.Http.Successful.OK
member JsonValue.ToString : saveOptions:JsonSaveOptions -> string
Published: Wednesday, 16 September 2015, 12:26 AM
Author: Tomas Petricek
Typos: Send me a pull request!
Tags: f#, web