Asynchronous client/server in F# (QCon 2012)

Qcon

Last week, I gave a talk on asynchronous programming in F# at London QCon 2012. The talk was a part of The Rise of Scala & Functional Programming track organized by Charles Humble. Reactive and asynchronous programming was a topic that was repeated a couple of times during the whole session - Sadek Drobi talked about non-blocking reactive web framework Play2 and Damien Katz talked about Erlang and CouchDB.

I used the one hour slot to implement "Rectangle Drawing App" - a simple application that shows how to write complete client-server application just using F#. On the server-side, I used asynchronous workflows to write HTTP server with an F# agent. On the client-side, I used asynchronous workflows to express user interface logic and the Pit project to run F# code as JavaScript that works everywhere. The app definitely had a huge commercial potential:

Quote

Asynchronous programming everywhere

In the F# asynchronous programming model, you can write code that does not block the executing thread, but can be written in the sequential style using standard control-flow constructs such as loops and exception handlers. This is quite different to what Play2 or Node.js use. These are essentially based on the event-based programming model. However, this comes with inversion of control and so we cannot use standard language features. With asynchronous programming in F#, you can write parallel code as well as sequential code.

Explaining the importance of asynchronous (non-blocking) code is quite easy. Just look at the following slide and you'll see why asynchronous programming should be used on both server-side and client-side:

To give a longer explanation, here are the key reasons:

  • On the server-side, asynchronous programming means that server is able to handle a large number of concurrent requests using fewer threads. When performing I/O operations (such as using file system or database), other requests can be processed on the same thread.

  • On the client-side, asynchronous programming makes it easier to express long-running interaction patterns. For example, drawing of a rectangle starts by pressing a button, continues by moving a mouse and ends by releasing of the button. Asynchronous programming gives you a way to directly encode this description.

I already wrote about both of these topics elsewhere, so if you want to learn more, look at the following articles:

Translating F# to JavaScript

For the client-side part of my demo application, I decided to use F# running as JavaScript using the excellent open-source F# to JavaScript translator called Pit. If you've been following my blog, you know that translating F# to JavaScript is a topic that I've been interested for a long time.

The Pit project is quite nice, because it does not try to do anything unnecessary. It lets you write code in F# and translates it to JavaScript. If you want some advanced abstractions for building web pages, you can implement them in F#. Pit project re-implements some of the standard F# libraries (e.g. for working with lists). In my demo, I wanted to use asynchronous workflows for GUI programming. Asynchronous workflows are not included in Pit, but it turns out that it was quite easy to re-implement them (see the source code at the end of the article).

When running as JavaScript, asynchronous workflows aren't useful for parallel programming, but they provide a great abstraction for making asynchronous AJAX calls and for implementing user-interface logic. To implement social rectangle drawing application, I needed to encode the following two state machines:

I won't explain the complete source code in this article. The rectangle drawing sample is already discussed in my book (Chapter 16) and in this book excerpt. The other workflow, which obtains the current state from the server and updates the displayed rectangles shows a few interesting aspects of writing client-side code in F# using Pit:

/// Asynchronously download retangles from the server
/// and decode the JSON format to F# Rectangle record
let [<Js>] getRectangles () : Async<Rectangle[]> = async {
  let req = XMLHttpRequest()
  req.Open("POST", "/get", true)
  let! resp = req.AsyncSend()
  return JSON.parse(resp) }

/// Repeatedly update rectangles after 0.5 sec
let [<Js>] updateLoop () = async {
  while true do 
    do! Async.Sleep(500)
    let! rects = getRectangles()
    cleanRectangles()
    rects |> Array.iter createRectangle }

The source code from the talk includes a couple of browser-based asynchronous operations. The first one is AsyncSend method of XMLHttpRequest, which sends the request to the server and then waits (without blocking), until the response is received. This means that getRectangles is not going to block the browser, but we can still use sequential programming style to implement it. After receiving the response as a string, the function calls JSON.parse to parse a standard JSON formatted data into an array of F# record values of type Rectangle.

The updateLoop function is also interesting. We don't want to initiate a new request every 0.5 seconds (in case the server is still processing the previous request). With asynchronous workflows, this is quite easy - the body of the loop waits 0.5 seconds, then asynchronously calls the server and updates the GUI. The next iteration of the loop only starts after the previous update completes (which corresponds to the above diagram).

Audience participation

A great thing about creating a client/server demo is that people can actually try it live while I was finishing my talk. The fact that the client-side of the demo used just JavaScript means that everybody was able to try it - when I put the link on the slide, people opened the site in their smart phones (I could see a few iPhones around) and created the following piece of modern digital art:

Links and materials

If you want to play with the sample, create a client-side F# application running as JavaScript or look at the slides from the talk, here are some useful links:

namespace Pit
namespace Pit.Dom
namespace Pit.Async
namespace Pit.Javascript
namespace SocialDrawing
module Dom

from SocialDrawing
module Colors

from SocialDrawing
type Rectangle =
  {X1: int;
   Y1: int;
   X2: int;
   Y2: int;
   Color: string;}

Full name: Blog.Rectangle

  type: Rectangle
  implements: System.IEquatable<Rectangle>
  implements: System.Collections.IStructuralEquatable
  implements: System.IComparable<Rectangle>
  implements: System.IComparable
  implements: System.Collections.IStructuralComparable
Rectangle.X1: int
Multiple items
val int : 'T -> int (requires member op_Explicit)

Full name: Microsoft.FSharp.Core.Operators.int

--------------------
type int<'Measure> = int

Full name: Microsoft.FSharp.Core.int<_>

  type: int<'Measure>
  implements: System.IComparable
  implements: System.IConvertible
  implements: System.IFormattable
  implements: System.IComparable<int<'Measure>>
  implements: System.IEquatable<int<'Measure>>
  inherits: System.ValueType


--------------------
type int = int32

Full name: Microsoft.FSharp.Core.int

  type: int
  implements: System.IComparable
  implements: System.IFormattable
  implements: System.IConvertible
  implements: System.IComparable<int>
  implements: System.IEquatable<int>
  inherits: System.ValueType
Rectangle.Y1: int
Rectangle.X2: int
Rectangle.Y2: int
Rectangle.Color: string
Multiple items
val string : 'T -> string

Full name: Microsoft.FSharp.Core.Operators.string

--------------------
type string = System.String

Full name: Microsoft.FSharp.Core.string

  type: string
  implements: System.IComparable
  implements: System.ICloneable
  implements: System.IConvertible
  implements: System.IComparable<string>
  implements: seq<char>
  implements: System.Collections.IEnumerable
  implements: System.IEquatable<string>
type Js = ReflectedDefinitionAttribute

Full name: Pit.Js

  type: Js
  implements: System.Runtime.InteropServices._Attribute
  inherits: System.Attribute
val createRectangle : Rectangle -> unit

Full name: Blog.createRectangle
val rect : Rectangle

  type: Rectangle
  implements: System.IEquatable<Rectangle>
  implements: System.Collections.IStructuralEquatable
  implements: System.IComparable<Rectangle>
  implements: System.IComparable
  implements: System.Collections.IStructuralComparable
val el : DomElement

  type: DomElement
  inherits: DomObject
val createSelection : unit -> DomElement

Full name: SocialDrawing.Dom.createSelection
property DomElement.ClassName: string
property DomElement.Style: DomStyle
property DomStyle.BackgroundColor: string
val moveElement : DomElement -> int * int -> int * int -> unit

Full name: SocialDrawing.Dom.moveElement
val getRectangles : unit -> Async<Rectangle []>

Full name: Blog.getRectangles


 Asynchronously download retangles from the server
 and decode the JSON format to F# Rectangle record
Multiple items
namespace Pit.Async

--------------------
type Async<'T> = | Cont of (('T -> unit) -> unit)

Full name: Pit.Async.Async<_>

--------------------
type Async =
  class
    static member AwaitEvent : event:IEvent<'T> -> Async<'T>
    static member AwaitEvent : event1:IEvent<'T1> * event2:IEvent<'T2> -> Async<Choice<'T1,'T2>>
    static member Sleep : milliseconds:int -> Async<unit>
    static member StartImmediate : workflow:Async<unit> -> unit
  end

Full name: Pit.Async.AsyncTopLevel.Async
val async : AsyncBuilder

Full name: Pit.Async.AsyncTopLevel.async
val req : XMLHttpRequest
type XMLHttpRequest =
  class
    new : unit -> XMLHttpRequest
    member Abort : unit -> unit
    member GetAllResponseHeaders : unit -> string
    member GetResponseHeader : unit -> string
    member Open : methodType:string * url:string * async:bool -> unit
    member Send : data:string -> unit
    member SetRequestHeader : key:string * value:string -> unit
    member ReadyState : int
    member ResponseText : string
    member ResponseXML : System.Windows.Browser.ScriptObject
    member Status : obj
    member StatusText : string
    member OnReadyStateChange : (unit -> unit) with set
  end

Full name: Pit.Javascript.XMLHttpRequest
member XMLHttpRequest.Open : methodType:string * url:string * async:bool -> unit
val resp : string

  type: string
  implements: System.IComparable
  implements: System.ICloneable
  implements: System.IConvertible
  implements: System.IComparable<string>
  implements: seq<char>
  implements: System.Collections.IEnumerable
  implements: System.IEquatable<string>
member XMLHttpRequest.AsyncSend : unit -> Async<string>
module JSON

from Pit.Javascript
val parse : string -> 'a

Full name: Pit.Javascript.JSON.parse
val updateLoop : unit -> Async<unit>

Full name: Blog.updateLoop


 Repeatedly update rectangles after 0.5 sec
Multiple items
static member Async.Sleep : milliseconds:int -> Async<unit>

--------------------
static member Async.Sleep : millisecondsDueTime:int -> Async<unit>
val rects : Rectangle []

  type: Rectangle []
  implements: System.ICloneable
  implements: System.Collections.IList
  implements: System.Collections.ICollection
  implements: System.Collections.IStructuralComparable
  implements: System.Collections.IStructuralEquatable
  implements: System.Collections.Generic.IList<Rectangle>
  implements: System.Collections.Generic.ICollection<Rectangle>
  implements: seq<Rectangle>
  implements: System.Collections.IEnumerable
  inherits: System.Array
val cleanRectangles : unit -> unit

Full name: SocialDrawing.Dom.cleanRectangles
module Array

from Microsoft.FSharp.Collections
val iter : ('T -> unit) -> 'T [] -> unit

Full name: Microsoft.FSharp.Collections.Array.iter

Discuss on twitter, .
Send corrections via GitHub pull requests.