F# Webcast (III.) - Using Asynchronous Workflows

In this webcast, we'll look at improving the code for downloading and processing RSS feeds that I presented in the second part (if you didn't see earlier parts, the first one was an introduction to basic functional ideas). The previous part demonstrated how to use .NET libraries and we implemented a simple downloadUrl function for obtaining content of from the web and we've also seen how to load the data into an XML document object and how to filter items. In this part, we'll modify the code to run asynchronously and potentially in parallel. To use some of the functionality, you'll need to get FSharp.PowerPack.dll, which is available with the VS 2008 installation or as a separated download for VS 2010 [4].

Now that we have the first version of the code, we can start refactoring it. I'm using the term in a slightly vague meaning - we're of course going to change the behavior of the code. We'll wrap it into F# asynchronous workflow to run without blocking threads and we'll also run multiple downloads in parallel. However, this can still be viewed as refactoring in some sense, because we're not changing the core behavior of the code. As you can see from the webcast, these kinds of refactorings are also very nicely supported by F# syntax...

F# Webcast Series

This webcast series follows the usual F# development process where you start with experimenting and writing an initial version of the solution interactively using the F# interactive tool. In the later phase of the process, we modify the code to make it parallel and asynchronous and finally, we can also encapsulate it into a standard .NET library that's usable from C#. Here is a list of all webcasts in the series:

• Part I. - Introducing functional concepts
  The first part introduces functional programming principles such as immutability, recursion and functions that take other functions as parameter (higher order functions). This can all be demonstrated in C# 3.0, so we start with C# and then look how the same concepts look in F#. Finally, the first part also shows functions for working with lists in F#.
• Part II. - Using standard .NET libraries
  The second part demonstrates how we can use standard .NET libraries. It uses classes from System.Net and System.Xml to download content of a web page (RSS feed), load it into XML document and process it to find only posts that contain some specified keyword.
• Part III. - Downloading web pages asynchronously
  The third part shows how to make the code from the part II. better. It introduces F# asynchronous workflows that can be used for writing code that doesn't block a thread when waiting for the completion of some I/O request. This part also shows how to modify the code to download and process multiple feeds in parallel.
• Part IV. - Developing standard .NET libraries
  In the fourth part, we look how to encapsulate the functionality written in F# into classes. We'll finally create a project (rather than just use F# scripts) and we'll wrap the code we wrote into a .NET class. We'll also look how to compile the project into DLL and how to use it from a simple C# web application.

Downloads 

• Source code from the webcast (F# Script)
• Download the video (WMV format, 19.2MB)

• [1] Microsoft Visual Studio 2010 First Look [^] - Microsoft.Com
• [2] Visual Studio 2010 Beta1 with F# is now available, plus matching F# CTP Update for VS2008 [^] - Don Syme's WebLog
• [3] F# May CTP for Visual Studio 2008 [^] - Microsoft Downloads
• [4] Microsoft F# PowerPack for .NET 4.0 Beta1 [^] - Microsoft Downloads

Published: June 5, 2009 03:39

Bookmarks: FaceBook |

• RE: F# Webcast: Using Asynchronous Workflows by Michael Robin (6/7/2009 12:34:35 AM)
  

  Some clarification please... I keep seeing the words "parallel", "concurrent", "fork", etc., used in conjunction w/async workflows -- however, these words are not as apt as "co-routines" or "non-preemptive multitasking", correct.
  I thought the whole point was the async workflows create a coroutine-like wrappers (using modads) around async operations - not multi-threaded, parallel, operations.
  
  So, which is true in an Async.Parallel:
  (a) All items run on the same, one (calling?) thread, and take turns on the thread as each becomes runnable. (You do mentioned the thread-pool though.)
  (b) Each item runs on it's own threads, and the Async.Parallel just acts as a big spawn/join.
  
  If (b) is true, then it seems not much value has been added, as there are many ways to do this.
  
  (a) seems like what is going on here, and seems more valueable, as getting this same behavior "long-hand" requires chaning many async callbacks to the next computation in the chain.
  However, if (a) *is* what is happening, why isn't it explained this way (as a state machine on one thread with state-transitions on result callbacks) everywhere, rather than in terms that sound like multi-threading is happening?
  
  Also, having the Run() happen on the calling thread seems much more valuable than picking something from the threadpool - this way a GUI program can do an async workflow operation and the message-pump could work in-between async calls w/o dealing with cross-thread marshalling. (No?)
  
  thanks for your time -
  m

• RE: F# Webcast: Using Asynchronous Workflows by Michael Robin (6/7/2009 1:13:59 AM)
  

  Ok - I see - the advantage of using the workflows in this Parallel case vs. using a new thread for each operation and just having the thread block on sync. I/O is more efficient use of the threadpool (if the programmer decides to use the default thread pool at all), as a free TP thread can be scheduled between each "let!". Very subtle, and unclear (at least to me at this point) how to "set the dial" between the continuation-passing and thread-parallel happenings. If
  
  Can I set the max # of threads to use (where 1 means no actual thread-parallel concurrency would take place - all workflow items interleaving on one thread) and <= len items means I could use multiple threads/cores)?
  
  
  

• RE: F# Webcast: Using Asynchronous Workflows by Jt Gleason (6/7/2009 7:13:36 AM)
  

  Your SearchItem code has a bug in it. You don't pass the feed on to your download URL.
  
  http://tomasp.net/articles/fsharp-screencasts/fsharp3_en.fsx
  
  let searchItems(feed:string) = async {
  let! xml = downloadUrl("http://blogs.msdn.com/MainFeed.aspx?Type=AllBlogs")
  
  
  should be
  let searchItems(feed:string) = async {
  let! xml = downloadUrl(feed)
  
  Other than that, excellent screencast. I'm heartened to see the growth of functional programming in "the real world".

• RE: F# Webcast (III.) - Using Asynchronous Workflows by Klaus Hebsgaard (10/3/2009 5:06:36 PM)
  

  Excellent introduction to async workflows - thanks!