Creating interactive You Draw bar chart with Compost

For a long time, I've been thinking about how to design a data visualization library that would make it easier to compose charts from simple components. On the one hand, there are charting libraries like Google Charts, which offer a long list of pre-defined charts. On the other hand, there are libraries like D3.js, which let you construct any data visualization, but in a very low-level way. There is also Vega, based the idea of grammar of graphics, which is somewhere in between, but requires you to specify charts in a fairly complex language including a huge number of transformations that you need to write in JSON.

In the spirit of functional domain specific languages, I wanted to have a small number of simple but powerful primitives that can be composed by writing code in a normal programming language like F# or JavaScript, rather than using JSON.

My final motivation for working on this was the You Draw It article series by New York Times, which uses interactive charts where the reader first has to make their own guess before seeing the actual data. I wanted to recreate this, but for bar charts, when working on visualizing government spending using The Gamma.

The code for this was somewhat hidden inside The Gamma, but last month, I finally extracted all the functionality into a new stand-alone library Compost.js with simple and clean source code on GitHub and an accompanying paper draft that describes it (PDF).

In this article, I will show how to use Compost.js to implement a "You Draw" bar chart inspired by the NYT article. When loaded, all bars show the average value. You have to drag the bars to positions that you believe represent the actual values. Once you do this, you can click "Show me how I did" and the chart will animate to show the actual data, revealing how good your guess was. Before looking at the code, you can have a look at the resulting interactive chart, showing the top 5 areas from the 2015 UK budget (in % of GDP):

What to teach as the first programming language and why

The number of Google search results for the phrase "choosing the first programming language" at the time of writing is 15,800. This illustrates just how debated the issue of choosing the first programming language is. In this blog post, I will not actually try to answer the question posed in the title of the post. I will not discuss what language we should teach as the first one. Instead, I will look at a more interesting question.

I will investigate the arguments that are used in favour of or against particular programming languages in computer science curriculum. I am more interested in the kind of argumentation that is employed to support a particular choice than in the specific languages involved. This approach is valuable for two reasons. First, by looking at the argumentation used, we can learn what educators consider important about computer science. Second, understanding the motivations behind different arguments allows us to make our own debates about the choice of a programming language more informed.

The scope of this blog post is limited to the choice of the first programming language taught in an undergraduate computer science programmes at universities. This means that I will not discuss other important contexts such as choices at a primary or a secondary education level, choices for independent learners and choices in other university degrees that might involve programming.

Note that this blog post is adapted from an essay that I wrote as part of a Postgrduate Certificate for Higher Education programme at University of Kent, so it assumes less knowledge about programming than a typical reader of my blog has. This makes it accessible to a broader audience thinking about education though!

Write your own Excel in 100 lines of F#

I've been teaching F# for over seven years now, both in the public F# FastTrack course that we run at SkillsMatter in London and in various custom trainings for private companies. Every time I teach the F# FastTrack course, I modify the material in one way or another. I wrote about some of this interesting history last year in an fsharpWorks article. The course now has a stable half-day introduction to the language and a stable focus on the ideas behind functional-first programming, but there are always new examples and applications that illustrate this style of programming.

When we started, we mostly focused on teaching functional programming concepts that might be useful even if you use C# and on building analytical components that your could integrate into a larger .NET solution. Since then, the F# community has matured, established the F# Software Foundation, but also built a number of mature end-to-end ecosystems that you can rely on such as Fable, the F# to JavaScript compiler, and SAFE Stack for full-stack web development.

For the upcoming December course in London, I added a number of demos and hands-on tasks built using Fable, partly because running F# in a browser is an easy way to illustrate many concepts and partly because Fable has some amazing functional-first libraries.

If you are interested in learning F# and attending our course, the next F# FastTrack takes place on 6-7 December in London at SkillsMatter. We also offer custom on-site trainings. Get in touch at @tomaspetricek or email tomas@tomasp.net for a 10% discount for the course.

One of the new samples I want to show, which I also live coded at NDC 2018, is building a simple web-based Excel-like spreadsheet application. The spreadsheet demonstrates all the great F# features such as domain modeling with types, the power of compositionality and also how functional-first approach can be amazingly powerful for building user interfaces.

Learn F# at TechMesh and SkillsMetter

Autumn is a busy period and I already invited you to a couple of interesting events, but there are two more events that you definitely should not miss. In only two weeks, you can come to two-day Progressive F# Tutorials packed with tutorials for both F# beginners and experts. At the beginning of December, the TechMesh Conference comes with three days of talks about alternative (and future) technologies.

I'll be speaking at both Progressive F# Tutorials and TechMesh and I'm also doing a tutorial at TechMesh, so if you want to learn about F#, type providers in F# 3.0 and financial computing with F#, here are some talks that you should not miss...

Why type-first development matters

Using functional programming language changes the way you write code in a number of ways. Many of the changes are at a small-scale. For example, you learn how to express computations in a shorter, more declarative way using higher-order functions. However, there are also many changes at a large-scale. The most notable one is that, when designing a program, you start thinking about the (data) types that represent the data your code works with.

In this article, I describe this approach. Since the acronym TDD is already taken, I call the approach Type-First Development (TFD), which is probably a better name anyway. The development is not driven by types. It starts with types, but the rest of the implementation can still use test-driven development for the implementation.

This article demonstrates the approach using a case study from a real life: My example is a project that I started working on with a friend who needed a system to log journeys with a company car (for expense reports). Using the type-first approach made it easier to understand and discuss the problem.

In many ways, TFD is a very simple approach, so this article just gives a name to a practice that is quite common among functional and F# programmers (and we have been teaching it at our F# trainings for the last year)...

F# Courses and Talks for Autumn 2012 (I.)

Similarly to the last year, I already have a number of F# events planned for the end of the summer and autumn that I'd like to invite you to!

The Visual Studio 2012 has been completed recently and it comes with F# 3.0. For me, this means two things. Firstly, it is the second Visual Studio version of F#, which means that functional programming is worth taking seriously. Secondly, F# 3.0 comes with type providers, which is a killer feature for working with data. No matter if you're a C# programmer now to functional programming or if you're an F# user in the real-world, I hope you can find some interesting and useful event below!

The two main things that I'm going to be involved in are SkilsMatter trainings in London and New York and a few events at the biggest functional conference (ICFP) in Copenhagen...

Asynchronous client/server in F# (QCon 2012)

Last week, I gave a talk on asynchronous programming in F# at London QCon 2012. The talk was a part of The Rise of Scala & Functional Programming track organized by Charles Humble. Reactive and asynchronous programming was a topic that was repeated a couple of times during the whole session - Sadek Drobi talked about non-blocking reactive web framework Play2 and Damien Katz talked about Erlang and CouchDB.

I used the one hour slot to implement "Rectangle Drawing App" - a simple application that shows how to write complete client-server application just using F#. On the server-side, I used asynchronous workflows to write HTTP server with an F# agent. On the client-side, I used asynchronous workflows to express user interface logic and the Pit project to run F# code as JavaScript that works everywhere. The app definitely had a huge commercial potential:

F# courses and talks (Winter 2012 and beyond...)

A couple of months ago, I posted a list of my F# talks and courses for Autumn 2011. Although I tried hard to have fewer speaking engagements during the winter and spring, there are quite a few events that I'd like to invite you to.

Last year, I spent quite a lot of time talking about asynchronous programming and agents. I think this is still a very important topic and especially agent-based programming in F# is a powerful way to implement concurrency primitives (like blocking queue), as well as complex systems (like trading screens and market analysis). I also wrote a series of articles on this topic that are available on MSDN and should be a good starting point.

Over the next few months, I'll be doing some talks about type providers, which is an upcoming F# 3.0 technology for accessing data. However, I also hope to find some time to look at other directions for F#, especially how it can be used in an online web-based environment, either using Azure or by translating F# to JavaScript using a recently announced open-source project named Pit.

Continue reading to see the list of planned talks, tutorials and courses....

Regions and navigation bar for F# in Visual Studio

The beginning of a new year may be a good time for writing one lightweight blog post that I wanted to publish for some time now. During my last internship with the F# team at MSR, I did some work on improving the F# IntelliSense in Visual Studio. This mostly involved the usual features - automatic completion, colouring and tooltips. The F# IntelliSense in Visual Studio 2010 still isn't perfect, but I think it is safe to claim that it is the best IDE experience for a typed functional programming language (aside, you can vote for some F# IDE features here and here).

F# Math (IV.) - Writing generic numeric code

Generic numeric code is some calculation that can be used for working with multiple different numeric types including types such as int, decimal and float or even our own numeric types (such as the type for clock arithmetic from the previous article of the series). Generic numeric code differs from ordinary generic F# code such as the 'a list type or List.map function, because numeric code uses numeric operators such as + or >= that are defined differently for each numeric type.

When writing simple generic code that has some type parameter 'T, we don’t know anything about the type parameter and there is no way to restrict it to a numeric type that provides all the operators that we may need to use in our code. This is a limitation of the .NET runtime and F# provides two ways for overcoming it.

• Static member constraints can be used to write generic code where the actual numeric operations are resolved at compile-time (and a generic function is specialized for all required numeric types). This approach makes resulting code very efficient and is generally very easy to use when writing a function such as List.sum.
• Global numeric associations (available in F# PowerPack) give us a way to obtain an interface implementing required numeric operations dynamically at runtime. This approach has some runtime overhead, but can be used for complex numeric types (such as Matrix<'T>).
• Combination of both techniques can be used to implement complex numeric type that is generic over the contained numeric values and has only a minimal runtime overhead.

Static member constraints are a unique feature of F# that is not available in other .NET languages, so if you're interested in writing numeric code for .NET, this may be a good reason for choosing F#. In C# or Visual Basic, you would be limited to the second option (which can be implemented in C#). In dynamic languages (like IronPython), everything is dynamic, so numeric computations can work with any numeric type, but will be significantly less efficient. In the rest of the article, we look at the three options summarized above.

This article is a part of a series that covers some F# and F# PowerPack features for numerical computing. Other articles in this series discuss matrices, defining custom numeric types and writing generic code. For links to other parts, see F# Math - Overview of F# PowerPack.

F# Math (III.) - Defining custom numeric types

In this article, we define an F# numeric type for calculating in the modular arithmetic (also called clock arithmetic) [1]. Modular arithmetic is used for calculations where we want to keep a value within a specified range by counting in cycles. For example, a maximal value on clock is 12 hours. When we add 11 hours and 3 hours, the value overflows and the result is 2 hours. Aside from clocks, this numeric system is also essential in cryptography or, for example, in music.

This tutorial shows several techniques that are essential when defining any new numeric type in F#. Most importantly, you’ll learn how to:

• Define a numeric type with overloaded operators
• Define a numeric literal for constructing numbers of our new type
• Enable calculating with our type in F# lists and matrices
• Hide implementation details of a numeric type

We define type IntegerZ5 that implements modular arithmetic with modulus 5, meaning that valid values are in the range from 0 to 4 and we equip the type with operations such as addition and multiplication. When an operation produces a value that would be outside of the range, we adjust it by adding or subtracting the modulus (in our case 5). Here are some examples of calculations that we’ll be able to write:

2 + 1 = 3 (mod 5)
4 * 2 = 3 (mod 5)
List.sum [ 0; 1; 2; 3 ] = 1 (mod 5)

In the first case, we can perform the operation without any adjustments. In the second case, we multiply 4 by 2 and get 8 as the result, which is out of the required range. To correct it, we calculate the remainder after a division by 5 (written as 8 % 5 in F#), which gives us 3. Finally, the last example shows that we’d also like to be able to use our type with lists. If we add values 0, 1, 2 and 3, we get 6 which is adjusted to 1.

This article is a part of a series that covers some F# and F# PowerPack features for numerical computing. Other articles in this series discuss matrices, defining custom numeric types and writing generic code. For links to other parts, see F# Math - Overview of F# PowerPack.

F# Math (II.) - Using matrices for graph algorithms

In the previous article of this series, we looked at complex and BigRational, which are two numeric types that are available in F# PowerPack. Aside from these two, the PowerPack library also contains a type matrix representing a two-dimensional matrix of floating-point values.

In this article, you'll learn how to work with matrices in F#, using some of the functions provided by F# PowerPack. I'll demonstrate the library using an example that represents graphs using a, so called, adjacency matrix. If you're not familiar with this concept, you don't need to worry. It is quite simple and it will be clear once we look at an example. The matrix represents which vertices of a graph are connected with other vertices by an edge. Many of the standard operations on matrices are useful when working with adjacency matrix, so this tutorial will cover the following:

• Creating matrices from lists and using functions from the Matrix module
• Using slices to read or modify a part of matrix
• Performing standard operations with matrices such as transposition and matrix multiplication
• Using higher order functions for working with matrices

This article is a part of a series that covers some F# and F# PowerPack features for numerical computing. Other articles in this series discuss matrices, defining custom numeric types and writing generic code. For links to other parts, see F# Math - Overview of F# PowerPack.

F# Math (I.) - Numeric types in PowerPack

In this article, we'll briefly look at two numeric types that are available in F# PowerPack. The type complex represents complex numbers consisting of real and imaginary parts. Both parts are stored as a floating point numbers. The type BigRational represents rational numbers consisting of numerator and denominator of arbitrary sizes. Integers of arbitrary size are represented using BigInteger type that is available in .NET 4.0 (in the System.Numerics.dll assembly). On .NET 2.0, the BigInteger type is also a part of F# PowerPack.

This article is a part of a series that covers some F# and F# PowerPack features for numerical computing. Other articles in this series discuss matrices, defining custom numeric types and writing generic code. For links to other parts, see F# Math - Overview of F# PowerPack.

F# Math - Numerical computing and F# PowerPack

This article is the first article of a series where I'll explain some of the F# features that are useful for numeric computing as well as some functionality from the F# PowerPack library. Most of the content was originally written for the Numerical Computing in F# chapter on MSDN (that I announced earlier), but then we decided to focus on using F# with third party libraries that provide more efficient implementation and richer set of standard numeric functionality that's needed when implementing machine learning and probabilistic algorithms or performing statistical analysis. If you're interested in these topics, then the last section (below) gives links to the important MSDN articles.

However, F# PowerPack still contains some useful functionality. It includes two additional numeric types and an implementation of matrix that integrates nicely with F#. The series also demonstrates how to use features of the F# language (and core libraries) to write numeric code elegantly. In particular, we'll use the following aspects:

• Overloaded operators. Any type can provide overloaded version of standard numeric operators such as +, -, * and / as well as other non-standard operators (such as .*). As a result, libraries can implement their own numeric types which are indistinguishable from built-in types such as int.
• Numeric literals. F# math libraries can enable using new numeric literals in the code. For example, you can write a BigRational value representing one third as 1N/3N. The N suffix used in the notation is not hardcoded in the F# language and we'll see how to define similar numeric type.
• Static constraints. F# supports static member constraints, which can be used for writing functions that work with any numeric type. For example, the List.sum function uses this feature. It can sum elements of any list containing numbers.

These are just a few of the F# language features that are useful when writing numeric code, but there are many others. The usual F# development style using interactive tools, type safety that prevents common errors, units of measure as well the expressivity of F# make it a great tool for writing numeric code. For more information, take a look at the MSDN overview article Writing Succinct and Correct Numerical Computations with F#.

F# courses and talks (Autumn 2011)

The end of the summer holiday season is getting closer. Luckily, I was in Prague last week so I actually noticed there was summer this year!

After a few quiet months, the autumn is going to be quite busy. Microsoft's //build/ conference will reveal the future of software development for Windows, but great things are going on in the F# world too. Don Syme is going to talk about F# Information Rich Programming at Progressive F# Tutorials in November, which should reveal more about F# 3.0 and type providers!

I have quite a few speaking engagements planned already, so if you want to learn about functional programming in .NET (and become a better C# programmer) or about F# and asynchronous programming, here are a few events that you may be interested in...

Programming with F# asynchronous sequences

In F#, we can represent asynchronous operations that do not block threads and eventually return a value of type 'T using asynchronous workflows Async<'T>. Asynchronous workflows can be easily constructed using the computation expression syntax async { ... } and there are also a few combi­nators that express more advanced composition (such as parallel composition or fork-join parallelism).

Sometimes, we need to use asynchronous operations that return more than just one value. For example, when downloading data from the internet, we would like to create an asynchronous sequence that returns the data in chunks as they become available.

One way to represent asynchronous operations that produce multiple values is to use the IObservable<'T> type from .NET. This isn't always the best option though. Observables implement push-based model, which means that chunks of data are generated as quickly as possible (until all chunks are emitted). What if we wanted to take the chunks one-by-one after the previous chunk is processed?

In this article, I describe asynchronous sequences. An asynchronous sequence is a simple, yet very powerful concept based on asynchronous workflows. It follows the same core model: results are generated on demand and asynchronously. Unlike asynchronous workflows, asynchronous sequences can be called (on demand) to generate multiple values until the end of the sequence is reached.

I first discussed asynchronous sequences with Don Syme, Dmitry Lomov and Brian McNamara in an email thread a long time ago. Thanks to Don for enthusiasm about the concept and for the first implementation of some of the combinators!

Real-World F# Articles on MSDN

More than a year ago, Mike Stephens from Manning (who was also behind my Real-World Functional Programming book) asked me if I'd be interested in collaborating on a project for MSDN. The idea was to collaborate with Microsoft on creating some additional content for the official F# Documentation.

A few days ago, the new content appeared on MSDN, so I finally have an excuse for the recent lack of blogging! Although the content contains a large number of new articles that are not based on my book, you can find it in the MSDN section named after my book, right under Visual F#. If you can't wait to check it out, here are all the links:

• The Real-World Functional Programming section on MSDN.
• The source code for individual chapters is available on MSDN Code Gallery.
• I also published annotated TOC with source code links on my functional programming web site.

While working on the articles, I also wrote about a few topics that we didn't use in the final version. You'll see them on my blog in the next few days, as soon as I edit them into a blog post form. Continue reading for more information about individual chapters.

Safer asynchronous workflows for GUI programming

In the previous article, I discussed how to use F# asynchronous work­flows for creating reactive user-interfaces. One of the main concerns was to avoid blocking the GUI thread (to prevent the user-interface from freezing). The workflow shouldn't perform any CPU-intensive compu­tation when running on the GUI thread.

The standard F# library provides two ways to run a computation on a background thread from an asynchronous workflow. The StartChild operation starts an operation in the thread pool and returns a workflow that can be called using asynchronous (non-blocking) let! construct. The SwitchToThreadPool operation can be called using do! and resumes the rest of the workflow on a background thread.

When using the SwitchToThreadPool operation, we also need to eventually use SwitchToContext to transfer the execution back to the GUI thread (after completing the CPU-intensive calculations). In this article, I describe a variation of F# asynchronous workflows that keeps track of the running thread in the type of the computation. As a result, calling a workflow that should be executed on a GUI thread from a background thread is a compile-time error as opposed to failing at runtime.

Writing non-blocking user-interfaces in F#

F# asynchronous workflows are best known as a way to write efficient I/O operations or as an underlying mechanism of F# agent-based programming (using the MailboxProcessor type). However, they are also very useful for user-interface programming. I think this is a very interesting and important area, so I already wrote and talked about this topic - it is covered in Chapter 16 of my book (there is a free excerpt) and I talked about it at F#unctional Londoners meeting.

Many applications combine user-interface programming (such as waiting for an event asynchronously) with some CPU-intensive tasks. This article looks at an example of such application and I'll explain how to avoid blocking the user-interface when doing the CPU-intensive task. The article starts with an example that is wrong and blocks the user-interface when doing data processing. Then I'll show you two options for fixing the problem. The three most important functions from the standard F# library that I'll discuss are Async.StartChild and Async.SwitchTo­ThreadPool with Async.SwitchToContext.

This is the first article of a mini-series. In the next article, I'll demonstrate a simple wrapper for F# async that makes it more difficult to write wrong programs. The wrapper keeps the desired thread (GUI or background) in the type of the computations and code that would block the user interface will not type-check. But first, let's look at the example...

Beyond the Monad fashion (I.): Writing idioms in LINQ

Thanks to LINQ and Erik Meier, monads have become a fashionable topic in the C# developer community. Indeed, no serious developer conference on .NET can get away without having a talk on monads. The attractive thing about LINQ and monads is that the SelectMany operator roughly corresponds to the bind function that defines a monad. In practice, LINQ is used for working with collections of data (IEnumerable<T>), but you can also define bind (i.e. SelectMany) for some other data types and use the LINQ syntax for working with other types. You won't be really using the full LINQ syntax. You'll probably use just nested from clauses (for binding) and select at the end to return the result.

However, monads are not the only notion of computation that we can work with. More interestingly, they are also not the only notion of computation that you can encode using LINQ! In this article, I'll briefly introduce idioms (also called applicative functors), which is another useful abstract type of computations. Idioms can be used for a few things that cannot be done using monads.

A provocative summary of this article is: "Everyone who tells you that LINQ is a monad is wrong!"

The truth is that LINQ syntax can be used for encoding queries (obviously), monads (as you were told), but also for idioms as you'll learn today (and quite possibly for other types of computations). In this article, we look at a basic example, but I'll describe a more complex real-world scenario in the next blog post.

MonoDevelop & Teaching F# & QCon tutorial

Point of sale application from QCon tutorial

It appears that I have been doing a lot more talking than writing in the last two months. I'm hoping to change this direction and I have two articles almost ready, so stay tuned! I was also posting all my interesting F# snippets to fssnip.net, which has grown quite a bit since I announced it in the last blog post. Thanks to everybody who submitted their snippets already and I'm looking forward to more! By the way, you can now run snippets using tryfs.net by clicking at "Load in TryF#" button.

In the meantime, the queue with talk materials that I wanted to post on my blog has grown to 3. I talked about F# in MonoDevelop in the Mono room at FOSDEM in February, then I did an online talk for the Community for F# . Finally, this week, I did a tutorial on F# with Phil Trelford at QCon in London (to fill in for Michael Stal due to unexpected health reasons).

Before I move on to writing about my recent experiments with LINQ, you can find materials from all of the recent talks below...

F# in Education & Concurrency with Agents

November was quite a busy month for me. First, I traveled to Cambridge (the "new one" in Massachusetts :-)) to present my work on the F# plugin for MonoDevelop at the F# in Education workshop. Shortly after getting back to London, I started working on a presentation about Concurrent Programming with Agents at the F#unctional Londoners meeting.

Now that both of the events are over, I'd like to write a short summary and also finally publish my slides and demos. In addition, both of the events were recorded (thanks to External Research group at MSR and SkillsMatter), so if you missed them, you can watch the recording...

Both of the events were really great and I had a chance to meet quite a few interesting people. One of the things that make F# great is the community around it. I think that one unique aspect of the F# community is its diversity. The same language is appealing to high school teachers, academics and researchers as well as software developers and technical directors from the industry. This combination is really valuable for both sides. It helps to transfer ideas from research to practice it gives researchers clear picture of problems in the industry that deserve their attention.

Asynchronous C# and F# (III.): How does it work?

Some time ago, I started writing a series about the differences between the asynchronous model in F# (in Visual Studio 2010) and the asynchronous language extensions for C# proposed at PDC 2010. In the first article, I showed how both of the language features look and I highlighted a couple of differences. In the second article, I discussed the support for cancellation (available only in F#) and how the two models differ semantically (i.e. what are differences in the behaviour). However, I didn't talk about more techincal differences and, in particular, how is the asynchronous code compiled. We'll look at this topic today...

Although the C# asynchronous programming model is very similar to F# asynchronous workflows, the compilation looks quite different. The C# compiler uses a similar technique as when compiling iterators and creates a state machine, while the F# compiler uses an approach based on higher order functions (combinators) that's shared with (most) other computation expressions.

I won't discuss the syntax of F# asynchronous workflows or the C# asynchronous extensions in this article, so if you're new to one of these, here are links to other articles in this series:

• Asynchronous C# and F# (I.): Simultaneous introduction
• Asynchronous C# and F# (II.): How do they differ?
• Asynchronous C# and F# (III.): How does it work?
• Asynchronous C# and F# (IV.): Calling F# libraries from C# (not yet available)

Let's start by looking at the mechanism used in the C# compiler. If you already know how iterators work in C# 2.0, then you'll find it quite easy to understand. If you're not familiar with iterators, then don't worry - I'll make the explanation self-contained.

F# Parallel Extras (III.): Financial dashboard with cancellation

In this article we'll look at several improvements that can be done to the Financial dashboard example (originally from the Parallel Programming with Microsoft .NET book). When I was translating samples in the book from C# to F#, the sample struck me, because it looks like a perfect fit for F# asynchronous workflows (instead of the Task<T> type used in the C# version). I already talked about an alternative implementation based on asynchronous workflows. However that version was still following some of the programming patterns, from the original C# version, which are not idiomatic in F#. In this article, I'll talk about a few other improvements that we can make to the sample...

In the original version of the sample (in both C# and F#), we explicitly checked whether a cancellation token has been cancelled in every single operation. This was needed in C#, because tasks do not support cancellation automatically. However, F# asynchronous workflows make cancellation easier. They automatically check if the computation should be cancelled at the beginning and the end of every asynchronous call. Our first change will be to use this feature. Also, the original version propagates a null value when the computation is cancelling. In F# we don't need that and we'll only change the type of the overall result to option<T>, so that we can notify the user interface (but we don't need to propagate cancellation explicitly). Finally, the original version contained sequential implementation, but didn't provide any way of actually running it, so we'll do a bit of refactoring to make that sample actually useable.

F# Parallel Extras (II.): Agent-based blocking queue

In the previous article, we briefly introduced the BlockingQueueAgent<T> type and we used it to implement the pipeline pattern (from Chapter 7 of Parallel Programming with Microsoft .NET) using asynchronous workflows. The type was used to represent intermediate buffers with a limited size. In this article we'll take a look at the implementation of the type. The type implements a very useful pattern in agent-based parallel programming, so you can use it in your work, but it could be also interesting as a demonstration of the F# Agent<T> type (an recommended alias for the MailboxProcessor<T> type).

The BlockingQueueAgent<T> type is similar to BlockingCollection<T> from .NET 4.0. It has methods for adding and removing elements that block when the operation cannot be done (e.g. adding when the queue is full or removing when the queue is empty). The most important difference is that it can be used asynchronously. This means that when we call its operations form F# asynchronous workflow (using let! and do!), the operation will block the calling workflow, but it will not block any physical thread. We start by looking at the overall structure of the agent and then look at the body of the agent which implements its behavior (using a state machine)...

F# Parallel Extras (I.): Image pipeline using agents

In a recent blog post series, I wrote about parallel programming samples that accompany the Parallel Programming with Microsoft .NET book by patterns & practices group at Microsoft. The F# translation of the samples that I wrote about mostly followed the style used in the book, so it used patterns that are typical for C#. However, some of the samples can be written in F# in a more interesting way...

In this article, we'll take a look at agent-based implementation of the Image pipeline example (from chapter 7). A pipeline is a useful pattern if you need to process large number of inputs in parallel and the processing consists of multiple phases or steps. In the original implementation, the pipeline was implemented using BlockingCollection<T> and Task<T> types from .NET 4.0.

In this article, I'll show a version that uses F# agents and asynchronous workflows. We'll use a BlockingQueueAgent<T> type, which is discussed in another article. It represents a queue with limited capacity that asynchronously blocks the process that is adding values if there is no space in the buffer and blocks the process that reads values when there are no values. This type can be elegantly used to implement the pipeline pattern. In this article, we'll demonstrate it by writing a four-phase pipeline that processes images. As you'll see, the agent-based version of the code is very much easier to write and has similar performance as the original version.

Looking under the cover (How does it work?)

Languages with type-inference make it possible to write shorter code, because a smart compiler infers the types automatically. Even though we don't write the types, they are still there and are used to check whether our code doesn't contain (certain type of) bugs. However, types are not only useful for safety, but also when understanding code. When you see a function together with its type (especially higher-order functions) you can understand code much more easily than when you just see its name. When writing F# code, you can see all the types thanks to great integration for Visual Studio. However, what if you post a code sample to a web site or a blog? Many F# code snippets (including some on this blog) are just difficult to read without types.

A few weeks ago, I was talking with James Margetson (from the F# team) about the possible uses of some functionality provided by the F# compiler and he suggested that we could use them to solve the problem I described in the introduction. Wouldn't it be nice if you could place a mouse cursor over an identifier in an F# blog post and see the same type information as the one that would appear in Visual Studio?

Here is an example of what I'm talking about:

1: /// Asynchronously download the content of a web page
2: let downloadUrl url = (...)
3: 
4: /// Download specified pages and add their lengths
5: let getTotalLength urls = async { 
6:   let! texts = 
7:     urls |> List.map downloadUrl
8:          |> Async.Parallel
9:   return texts |> Seq.sumBy String.length }F# Web Snippets

If you cannot wait and want to create F# source code snippets like this yourself, then go straight to F# Web Snippets (please use the web based version carefully - processing is pretty demanding!). If you first want to know how it works and how to use it, as well as where to get the source code, then continue reading...

Parallel Programming in F# (IV.): Financial dashboard example

In the fourth part of the Parallel Programming in F# series, we'll take a look at the Adatum Financial Dashboard sample. The sample is a part of Parallel Programming with Microsoft .NET, which is a guide that introduces common parallel programming patterns on .NET 4.0. The C# version of the sample is in details discussed in the guide, but the F# translation contains several additional interesting aspects that I'll introduce in this article.

The sample simulates a financial modelling application that performs processing of market data in several steps that can be partly executed in parallel. In this article we'll compare two implementations. The first one (similar to the C# version) uses the Task<'T> type from .NET 4.0 and chains steps using the ContinueWith method. The second version is F# specific and it implements step as a sequential computation wrapped in asynchronous workflows. Partial results are reported to the user interface using events.

Parallel Programming in F# (III.): Aggregating data

In this part of the Parallel Programming in F# series, we'll explore examples of parallel aggregation from Chapter 4 of Parallel Programming with Microsoft .NET, which is a guide that introduces common parallel programming patterns on .NET 4.0. The C# version of the sample is in details discussed in the guide. In this article, we'll look at the F# translation and in particular at several functions from the PSeq module. Some of the functionality is currently available only in the "PSeq.fs" distributed with the samples, but will eventually appear in F# PowerPack as well.

Aggregation of data in parallel is an interesting problem. As we've seen in the previous article, PLINQ and tasks make it easy to parallelize independent blocks of code that don't share any state. Unfortunatelly, not all programs are like that. In particular, we often need to aggregate all elements of a sequence - when writing sequential code in F#, you would use the Seq.fold function. In this article, we'll look at functions that implement fold parallel.

Parallel Programming in F# (II.): Using PLINQ and Tasks

In this part of the Parallel Programming in F# series, we'll look at some basic examples of parallel programming in F# that are available in Parallel Programming with Microsoft .NET. This is a guide that introduces common parallel programming patterns on .NET 4.0. The guide discusses the samples in C#, but it also contains an F# translation of the source code. Since the languages are different, the F# version deserves a brief explanation.

In this article, we'll discuss some of the F# examples and we'll look at a couple of aspects that are specific for F#. In particular, we'll look at working with Parallel LINQ (both directly and using PSeq module from F# PowerPack) and working with tasks. We'll also look at an interesting example (using closures) where C# version doesn't behave as expected, but a literal translation in F# corrects the problem.

Parallel Programming in F# (I.): Introducing the samples

Parallel Programming with Microsoft .NET is a guide written by the patterns & practices group at Microsoft. It introduces .NET programmers to patterns for including parallelism in their applications (using support for parallel programming in .NET 4.0). It introduces techniques such as parallel loops, parallel tasks, data aggregation and so on. The book includes numerous samples in C# and Visual Basic that can be easily copied and adapted to match your needs.

As part of a contracting work for the F# team, I developed an F# version of the samples, which is now available on the book web site. You can get it by downloading F# code samples (ZIP) from the 1.0 release, or you can download the latest version of the source code directly. The F# version of the code is mostly a direct translation of the C# versions, but there are a few interesting places that are worth discussing. In particular, some of the samples use the PSeq module from F# PowerPack and I also added a version of one example that uses F# asynchronous workflows.

In this article series, I'll look at several interesting examples from the F# version of the source code. We'll look how to use PLINQ and Tasks (which are available in .NET 4.0) from F# (Part II.) including some advanced topics such as the Map-Reduce algorithm (Part III.). We'll also look at a larger example built using tasks and an alternative implementation using asynchronous workflows (Part IV.) Here are links to the individual articles:

• Parallel Programming in F# (I.): Introducing the samples
• Parallel Programming in F# (II.): Using PLINQ and Tasks
• Parallel Programming in F# (III.): Aggregating data
• Parallel Programming in F# (IV.): Financial dashboard example

Dynamic in F#: Reading data from SQL database

When reading data from a SQL database in F#, you have a couple of options. You can use F# implementation of LINQ, which allows you to write queries directly in the F# language or you can use standard ADO.NET classes such as SqlCommand. The second option is often good enough - when you just need to call a simple query, you probably don't want to setup the whole LINQ infrastructure (generate classes, reference F# PowerPack, etc.) Unfortunately, ADO.NET classes are a bit ugly to use. When calling an SQL stored procedure, you need to specify the name of the procedure as a string, you need to add parameters one-by-one by creating instances of SqlParameter, you need to dynamically cast results from the obj type and so on...

In this article, we'll look how to use the dynamic operator to make the experience of using ADO.NET from F# dramatically better. Dynamic operator (there are actually two of them) are a simple way of supporting dynamic invoke in F#. We can use it to write code that looks almost like an ordinary method call or property access, but is resolved dynamically at runtime (using the name of the method or property). The following example shows what we'll be able to write at the end of this article:

// Call 'GetProducts' procedure with 'CategoryID' set to 1
use conn = new DynamicSqlConnection(connectionString)
use cmd = conn?GetProducts
cmd?CategoryID <- 1
conn.Open()

// Read all products and print their names
use reader = cmd.ExecuteReader()
while reader.Read() do
  printfn "Product: %s" reader?ProductName

If you ever tried to call a SQL stored procedure directly using the SqlCommand, then you can surely appreciate the elegance of this code snippet. Let's now take a look at a larger example and some of the neat tricks that make this possible...

Recording and samples from my Reactive F# talk

Almost a week ago, I posted an invitation to my F# talk at the F#unctional Londoners user group. The theme of my talk was reactive programming, which was also a topic of my Master's thesis (defended earlier in June), so I really enjoyed talking about it. In the talk, I discussed the two approaches that you can use for developing reactive applications in F# (using examples in Silverlight):

• Declarative (or data-flow oriented) allows you to describe "what" should be done with data that your component receives from various events (such as mouse position etc.) This can be written using F# event combinators such as Event.map and Event.scan.
• Imperative (or control-flow oriented) - in this style, we describe various states of the component (e.g. semaphore with green, orange and red) and describe transitions between the states. This can be written using F# asynchronous workflows and the AwaitObservable primitive (which you can get as part of the source code).

Thanks to the folks from SkillsMatter who provided place for the meetup and helped with the organization, the talk was also recorded and is already available online! Below, you can also get all the nice Silverlight demos that I used during the talk...

Thanks again to Carolyn Miller and Phil Trelford for organizing the talk and also to Don Syme for taking me to the airport in the early morning (at 4 AM...) after the talk.

Links & Resources

• Reactive Programming in F# (recording) - Skills Matter
• Slides from the talk (PDF, PPTX formats)
• Reactive F# demos in Silverlight (includes AwaitObservable implementation)

Reactive Programming with F# in London

If you’re a registered member of the F#unctional Londoners user group, then you maybe already know that I'll be visiting London on June 23 and I’ll be talking about Reactive programming with F#. If you're not a registered member and occasionally visit London, then you should definitely register. The user group is organized by Carolyn Miller and Phil Trelford (whom I met some time ago at Microsoft Research in Cambridge). Among others, previous speakers include Robert Pickering (who also presented some samples based on my F# and Accelerator series). Finally, another reason for joining the group is that it has a great name (as well as a logo)!

When, what & where?

• Tomas Petricek on Reactive Programming with F#
• Date & time: 23 June (Wednesday), 6:30 PM
• Location: The Skills Matter eXchange, 116-120 Goswell Road, London

By the way, I'll also have a free copy of my Real-World Functional Programming book to give away during the talk!

Reactive Programming with F#

I'm sure you're already convinced to come. Nevertheless, you may still want to know what I'm going to talk about. There are many areas where F# offers clear benefits such as parallel & concurrent programming. I believe that reactive programming is another great area for F#. In reactive programming, we face quite different problems than in other programming styles. We (as the authors of the application) can no longer specify what the application should do. Instead, the application needs to be able to handle many possible combinations of events. This aspect of programming is sometimes called inversion of control.

Reactive programming is important for programming user interfaces, especially today when user interfaces are becoming more interactive and more "fancy". To demonstrate this, I'm working on some nice Silverlight demos for the talk! However, it is also needed to handle other kinds of events such as completion of background task or message from other application. We'll look at the two essential techniques that F# provides for reactive programming:

• Declarative event combinators - one way of writing reactive applications is to specify the whole event processing declaratively by saying "what" should be done with occurrences of events. This is particularly useful when we need to encode simpler logic with a clear data-flow.
• Imperative using workflows - for more complicated interactions, we can use asynchronous workflows. This makes the code more explicit, but we get full control over the control-flow of the application. Even though this approach is more "imperative" it can be used for writing nicely composable functional code as well.

I'm looking forward to seeing you at the talk next week!

ASP.NET and F# (I.) - Creating MVC web applications in F#

Some time ago, I wrote a couple of examples of developing web applications in F# using ASP.NET. Since then, the F# language and runtime has changed a little bit and there are also new technologies available in ASP.NET, so I thought I'd write a more up-to-date article on this topic. In this article, I'll present a simple "demo" F# web application that you can use as a starting point when creating your own projects (you'll also find a convenient Visual Studio 2010 template below). The article shows the following interesting things:

• ASP.NET MVC - We're going to use ASP.NET MVC Framework to create the web application. As the article name suggests, most of the actual program code including models and controllers will be implemented in F#.
• F# LINQ to SQL - The application uses a sample Northwind database and we'll write queries for selecting data from the database using LINQ support that's available in the F# PowerPack.
• F# features - The application also uses some nice F# features that are quite useful for developing web applications. We'll use modules and records to implement the model and we'll also use advanced meta-programming features for constructing LINQ queries.

If you want to use F# for creating an MVC application, you have a few options. It should be possible to create the web application solely as an F# project. However, we'll use a more convenient approach. We'll create a standard C# MVC project and move all the actual implementation to an F# library. We'll look at the application structure shortly. The following screenshot shows a page that lists products in the Northwind database:

Programming user interfaces using F# workflows

Numerous Manning partners already published several exceprts from my Real-World Functional Programming book. You can find a list on the book's web page. However, the last excerpt published at DotNetSlackers is particularly interesting. It discusses how to use F# asynchronous workflows to write GUI applications. This is a very powerful programming pattern that is very difficult to do in any other .NET language. We first discussed it with Don Syme during my internship at Microsoft Research and I found it very elegant, so I made some space for it in the book. In fact, the entire Chapter 16 discusses various reactive programming techniques that can be used in F#.

When designing applications that don't react to external events, you have lots of control flow constructs available, such as if-then-else expressions, for loops and while loops in imperative languages, or recursion and higher-order functions in functional languages. Constructs like this make it easy to describe what the application does. The control flow is clearly visible in the source code, so drawing a flowchart to describe it is straightforward.

Understanding reactive applications is much more difficult. A typical C# application or GUI control that needs to react to multiple events usually involves mutable state. When an event occurs, it updates the state and may run more code in response to the event, depending on the current state. This architecture makes it quite difficult to understand the potential states of the application and the transitions between them. Using asynchronous workflows, we can write the code in a way that makes the control flow of the application visible even for reactive applications.

You can read the complete article here: Programming user interfaces using F# workflows [^]. It is also worth adding that Manning offers 30% discount to DotNetSlackers readers (see the article for details!)

Deal of the day: Real-World Functional Programming

Some time ago, I received my copies of Real-World Functional Programming. I started working on it back in May 2008 and as many people who had more experience with writing books told me, it took longer than I was expecting! Anyway, I have to say, it was worth it, holding the actual printed book with my name on the cover is just fantastic!

The goal of the book is to present functional programming concepts and ideas in a readable form. I wanted to create a book that will teach you how to think functionally without using the usual shock therapy that people usually feel when seeing functional programming for the first time. There are already a couple of reviews that suggest I was quite successful:

• Functional Programming for the Real World, by Tomas Petricek and Jon Skeet, has been a very helpful book for moving to F# from C#, as the authors do a fantastic job of helping to explain the differences between OOP and FP.
  James Black at Amazon.com
• This book isn’t just a simple introduction to programming in F#; it’s an introductory text on functional programming covering the many reasons why it is time for this programming paradigm to finally be accepted by mainstream programmers. And it also contains much more...
  CliveT, Software Engineer at Red Gate Software
• ... and there are many other great comments about the book at Manning book page.

Deal of the day (January 24)

Finally, here is one great news if you're interested in getting the book! Real-World Functional Programming is Manning's Deal of the Day this Sunday, January 24. On this day, the print book is available for $20 from the Manning website, with code dotd0124.

Accelerator and F# (IV.): Composing computations with quotations

In this article series, we're talking about the Accelerator project and I'm presenting an F# library that I implemented, which allows you to use Accelerator [references] in a more sophisticated way. We've seen two examples of using Accelerator directly (see also introduction and Game of Life). In the previous article I introduced my F# library for working with Accelerator. We've seen F# functions from the DataParallel module, we implemented an algorithm that rotates an image using these functions and finally, we've seen that we can take this ordinary F# code and run it using Accelerator. This is all possible thanks to F# quotations, which we can use to get an AST (a source code) of an F# function we wrote (if the function is marked in some special way).

In this part of the series, we're going to look at working with quotations explicitly. We'll use meta-programming techniques to work with Accelerator. Meta-programming means writing programs that manipulate with other programs or pieces of code. This is exactly what we're going to do in this article. We'll write an F# function (running on CPU) that builds a program, which we'll then run using Accelerator.

This is quite interesting approach, which isn't possible when we call Accelerator methods as standard F# functions or .NET methods. The benefit is that we'll clearly see which parts of program run on CPU and what parts execute on GPU or using X64 multi-core target. We could also perform more complicated optimizations with the code (because this wouldn't affect the readability). Just for your reference, here is the list of articles in this series in case you missed some of them:

• Accelerator and F# (I.): Introduction and calculating PI
• Accelerator and F# (II.): The Game of Life on GPU
• Accelerator and F# (III.): Data-parallel programs using F# quotations
• Accelerator and F# (IV.): Composing computations with quotations

However, enough with theory and let's take a look at some code samples! This time, we'll implement blurring of an image (also called convolution). Another example how to write this in F# using Accelerator is Satnam Singh's blog post [4]. Our example will be different, because we'll write the code as standard F# program and then have it translated to Accelerator automatically using quotations. We'll also talk about the architecture of the library that we're using and look at some performance results.

Accelerator and F# (III.): Data-parallel programs using F# quotations

If you've been following this article series, you already know that Accelerator is a MSR library [1, 2] that allows you to run code in parallel on either multi-core CPU or using shaders on GPU (see introduction). We also discussed a direct way to use Accelerator from F# (by calling Accelerator methods directly) and implemented Conway's Game of Life. In this article, we'll look at more sophisticated way of using Accelerator from F#. We'll introduce F# quotations and look at translating 'normal' F# code to use Accelerator.

In general, F# quotations allow us to treat F# code as data structure and manipulate with it. This is very similar to C# expression trees, but the F# implementation is more powerful. We can also mark a standard method or a function with a special attribute that tells the compiler to store quotation of the body. Then we can access the quotation and traverse it or modify it. In this article we'll use a function that takes an F# quotation (containing a limited set of functions) and executes it using MSR Accelerator. Implementing this functionality is a bit complicated, so we won't discuss the implementation now. We'll leave this for some future article of this series. In future, we'll also look at other interesting possibilities that we have when writing code using quotations. Here is a list of articles in this series and of the articles that I'm planning to add:

• Accelerator and F# (I.): Introduction and calculating PI
• Accelerator and F# (II.): The Game of Life on GPU
• Accelerator and F# (III.): Data-parallel programs using F# quotations
• Accelerator and F# (IV.): Composing computations with quotations

Accelerator and F# (II.): The Game of Life on GPU

In the previous article, I introduced the Microsoft Research Accelerator library. It allows us to write computations with arrays in C# and execute them in parallel on multi-core CPU or more interestingly, using GPU shaders. In the previous artcile, we've seen how Accelerator works and how it can be accessed from F#. In this article, we'll look at one more interesting F# demo - we'll implement the famous Conway's Game of Life [1] using Accelerator. We'll use a v2 version of Accelerator which has been announced just recently and is available from Microsoft Connect [2].

This article is the second one from a series about using Accelerator from F#. Today, we'll use Accelerator types directly from F# - this is the simplest possible approach and is very similar to the way you'd work with Accelerator in C#. However, we can use some nice F# features such as custom operators to make the code more readable. In the next article, we'll discuss a different approach - we'll look how to execute more "standard" F# code (that doesn't reference Accelerator explicitly) with Accelerator using F# quotations. The list of articles may change, but here is a list of articles that I'm currently planning to write:

• Accelerator and F# (I.): Introduction and calculating PI
• Accelerator and F# (II.): The Game of Life on GPU
• Accelerator and F# (III.): Data-parallel programs using F# quotations
• Accelerator and F# (IV.): Composing computations with quotations

Accelerator and F# (I.): Introduction and calculating PI

I already wrote about two projects that I worked on during an internship at MSR back in 2007 (ASP.NET support in F# and F# WebTools). Even though this was more than 2 years ago (and I did one more internship at MSR in the meantime), I still have one more project that I never published on the web. The folks from the F# team reminded me of this project recently, so I thought I could finally publish it. The project used Microsoft Research Accelerator [1, 2], which is a C# library for developing array-based computations and executing them on a GPU. More recently, the Accelerator team at MSR published Accelerator v2 [3], which was a good motivation to update my original project...

In this article, we'll look at the simplest way of using Accelerator from F#. Accelerator provides a managed interface that can be naturally used from both C# and F#. We can use a mix of method calls and overloaded operators to describe a computation. In F#, we'll also define our additional custom operators to make the code a bit nicer. After we introduce Accelerator using a simple C# demo, we'll look how to calculate an approximate value of the PI number using a Monte-Carlo method.

This article is the first one from a series about using Accelerator from F#. The list of articles may change, but here is a list of articles that I'm currently planning to write:

• Accelerator and F# (I.): Introduction and calculating PI
• Accelerator and F# (II.): The Game of Life on GPU
• Accelerator and F# (III.): Data-parallel programs using F# quotations
• Accelerator and F# (IV.): Composing computations with quotations

Real-World Functional Programming: Completed and printed!

If you're following my blog or if you're interested in F# or functional programming in .NET, you probably noticed that I was working on a book Real-World Functional Programming. At some point, we called it Functional Programming for the Real-World, but then we changed the title back to a better sounding version Real-World Functional Programming (subtitle With examples in F# and C#). The book is also reason for a lower number of blog posts over the last year. Over the last month or so, we were doing the final edits, reviewing the final PDF version (I fixed quite a lot minor issues, synchronized book with the Beta 2 F# release and so on). Anyway, before a few days, I received the following email (as an author, I receive the same emails as those who ordered the book through the Manning Early Access Program, so that I can see what we're sending to our dear readers):

Dear Tomas Petricek, We are pleased to announce that Real-World Functional Programming is now complete! As a MEAP subscriber you can download your copy of the finished ebook right now! (...) This ebook is the final version, identical to the softbound edition, which is currently being printed and will be available on December 24. If you chose the printed book option when you originally subscribed, we'll ship it to you automatically—no action required from you.

Finally finished!

Yes, that's right. The book is finally completed and as far as I know, it has been printed last week! If you already ordered the book, you won't receive it before Christmas, but it should come shortly after. I can't wait to see the book actually printed. The transition from the Word drafts I initially wrote to a final PDF version was already felt fantastic and I thought "It looks like a real book!" Among other things, there are now graphical arrows with comments inside listings, which looks really great and makes code listings much easier to read. Now I can look forward to seeing the actual book. Maybe I'm too conservative, but I have to say that I'm really glad that I wrote the book before everything is going to be published just electronically!

Here is a couple of links that you may found interesting if you want to look inside the book...

Functional Programming: Available Chapter Excerpts & Discount

The work on my book Functional Programming for the Real World is slowly getting to the end. I'm currently creating index for the last couple of chapters and doing final updates based on the feedback from reviews and also from the forum at manning.com (this means that if you have some suggestions, it's the best time to post them - I haven't yet replied to all of them, but I'll certainly do that before the manuscript will go to the production).

F# Webcast (IV.) - Developing standard .NET libraries

In the previous parts of this webcast series we've developed an F# script that downloads RSS feeds asynchronously and in parallel and searches them for the specified keywords. We followed the usual F# development style, so after introducing the basic functional concepts, we wrote the code in the simples possible style and demonstrated how to use System.Xml and System.Net namespaces. Then we refactored the existing code, to run asynchronously and process the results potentially in parallel, which was very easy thanks to F# asynchronous workflows.

In this part of the series, we'll make the next evolutionary step of our sample application. We'll turn the code that originally used F# tuples and lists into code that uses standard .NET objects and we'll also see how to declare a class in F#. This simple modification will turn the script into an F# library that is almost indistinguishable from a library developed in C#. We'll also look how you can use the library from C# web application to show the interop between C# and F# in practice. We'll start with the code from the previous part, so if you missed that, you may want to check it out or download the source code.

F# Webcast (III.) - Using Asynchronous Workflows

In this webcast, we'll look at improving the code for downloading and processing RSS feeds that I presented in the second part (if you didn't see earlier parts, the first one was an introduction to basic functional ideas). The previous part demonstrated how to use .NET libraries and we implemented a simple downloadUrl function for obtaining content of from the web and we've also seen how to load the data into an XML document object and how to filter items. In this part, we'll modify the code to run asynchronously and potentially in parallel. To use some of the functionality, you'll need to get FSharp.PowerPack.dll, which is available with the VS 2008 installation or as a separated download for VS 2010 [4].

Now that we have the first version of the code, we can start refactoring it. I'm using the term in a slightly vague meaning - we're of course going to change the behavior of the code. We'll wrap it into F# asynchronous workflow to run without blocking threads and we'll also run multiple downloads in parallel. However, this can still be viewed as refactoring in some sense, because we're not changing the core behavior of the code. As you can see from the webcast, these kinds of refactorings are also very nicely supported by F# syntax...

F# Webcast (II.) - Using .NET libraries

About a week ago I posted the first part of my F# webcast series. It focused on explainining the basic ideas behind functional programming such as immutability, recursion and passing functions as arguments to other functions (or methods in C#). In the first part, we've seen some C# code to demonstrate the ideas and also a bit of F#, mainly to show the basic language features.

The second part is going to be exclusively about F#. It'll demonstrate how we can start writing a demo application that grabs data from RSS feeds and processes them. You'll learn how to access .NET libraries from F# (in particular, we'll use System.Net and System.Xml). We'll develop the code iteratively, which means that we'll start by simply enumerating the RSS elements using for loop and printing the results and then we'll refactor the code to use tuples and sequence expressions to turn it into processing code that generates a sequence of feed items. Finally we'll also demonstrate how to use some of the functions from the previous part such as List.filter in practice.

F# Webcast (I.) - Introducing functional concepts

Now that Visual Studio 2010 Beta 1 is out, it is finally a good time to take a look at one of the (in my opinion) most interesting new features in the new release - the F# language. F# existed for quite a long time now as Microsoft Research project, but is now becoming a real Microsoft product. Interestingly, F# is still available as a plugin for Visual Studio 2008, so if you want to try it you don't have to install the whole new beta of 2010.

There are already many resources for learning F# including my functional programming overview, which is a Manning Greenpaper for the book Functional Programming for the Real World that I'm writing with Jon Skeet and my four-part F# introduction. There are also some useful links on the official F# web site including some talk recordings. However, I haven't yet seen any good F# webcast focusing mainly on showing F# source code, starting from simple functional concepts to the real-world features like asynchronous workflows and object-oriented programming in F#, so I decided to create one.

So, here it is...

Imperative computation in F# (II.) - Writing break and continue

As I already wrote in the first part of this series, the F# language doesn't support some of the language constructs known from imperative languages such as C#. In particular, we cannot use imperative return statement that returns the result of a function from any place in the function code. In functional languages, every construct is an expression, so to get the overall result of the function, the F# language evaluates the expression and the value of the expression is used as the result. In the previous article, we've seen that we can simulate this construct in the F# language using F# computation expressions and I showed how to implement computation named imperative that allows us to write for example the exists function for working with sequences like this:

let exists f inp = imperative {
  for v in inp do 
    if f(v) then return true
  return false }

In this article, we're going to look at two more imperative constructs and we're going to talk about break and continue. We'll see that we can quite easily extend the computation builder from the previous article to allow writing code that is syntactically very close to what you would write in C#. As I already mentioned, there are of course some performance overheads when using computation expressions, but I find it very interesting how nice imperative syntax we can get in functional F#:

imperative { 
  for x in 1 .. 10 do 
    if (x % 3 = 0) then do! continue
    printfn "number = %d" x }

The only difference between this code and the code we'd probably write if F# supported continue as a keyword is that we need to wrap the code inside the imperative computation and that we need to add the do! primitive before the continue value. Now that we've seen an example of using the continue value inside the imperative computations, let's look how we can extend the computation builder from the previous article to add this feature...

Imperative computation in F# (I.) - Returning results from a function

One of the limitations of F# is that it doesn't very well support some of the advanced imperative language constructs such as break, continue or imperative style of returning value from a function, meaning that you can't write something like return false in the middle of the function. This has good reasons. F# doesn't in principle have the notion of currently executing statement and instead treat every code you write as an expression. Clearly, when there is no current statement, we cannot jump to other statements. If you're looking for more information about these basic principles, you can take a look at my book Real World Functional Programming, which covers this distinction in details in chapter 2, but we'll look at a brief example that will clarify this idea shortly.

Often, there is really no need to use break or other imperative constructs in F#, because you can write the same thing more elegantly using one of the provided higher order function such as Seq.exists or Seq.tryfind. However, there are still some cases where the imperative programming style makes it easier to express our original intention. Also, implementing your own higher order functions (akin to Seq.exists) would sometimes be much easier if we could just use imperative return.

So, what can be done about this?

Real World Functional Programming: Second review finished!

I've been working on the Real World Functional Programming in .NET book for quite some time now. In fact, I had the first discussions with Michael Stephens from Manning in March last year and I started thinking about the book at that time, so it has been occupying my mind for almost a year now! Until recently, I was feeling that we're not getting much closer to the end of this project, because writing a book is just a lot of work. However, I think I can finally see that we're getting closer to actually finishing the book. At Manning, we've recently finished the second review, which means that I've just got another set of very useful comments - a big thanks to all the reviewers! I'm also getting close to finishing the first draft of the whole manuscript (depending on the reviews, the content may still change a bit, but I expect to write at most one new chapter from now). Hopefully, the drafts will soon make it to the MEAP release of the book.

Source code for Real World Functional Programming available!

As you can see, there has been quite a bit of silence on this blog for a while. There are two reasons for that - the first is that I'm still working on the book Real World Functional Programming, so all my writing activities are fully dedicated to the book. The second reason is that I'm having a great time doing an internship in the Programming Principles and Tools group at Microsoft Research in Cambridge with the F# team and namely the F# language designer Don Syme. The photo on the left side is the entrance gate to the Trinity College of the Cambridge University taken during the few days when there was a snow. I recently started using Live Gallery, so you can find more photos from Cambridge in my online gallery. Anyway, I just wanted to post a quick update with some information (and downloads!) related to the book...

Functional Programming in .NET using C# and F# (Manning Greenpaper)

Functional programming languages have been around for a while and were always astonishing for their ability to express the ideas in a succinct, declarative way allowing the developer to focus on the essence of problem rather than on technical details of the solution. Recently, functional paradigm is gaining new prominence as an efficient way to handle development of multi-processor, parallel and asynchronous applications.

Functional ideas are arising in C# as well as in other main-stream languages and functional languages are becoming an alternative for real-world projects. Also, Microsoft recently introduced a new language called F#, which has a strong background in traditional functional languages, but as a .NET language also benefits from the rich .NET and Visual Studio ecosystem.

Available via MEAP | 500 pages
Softbound print: March 2009 (est.)

This article is partially an excerpt from my book Real-world Functional Programming in .NET [1]. Thanks to my editors at Manning I have the permission to publish it on my blog. We’ll look at several aspects of functional programming and how the same concepts, which are essential for the functional paradigm, look in the F# and in C# 3.0 with LINQ. We will shortly look at the basic programming language features like lambda functions and type inference that are now available in both F# and C#. Functional programming isn’t only about language features, but also about using different programming style, so we’ll look at some high level concepts as well. These include using immutable data structures for developing code that can be executed in parallel and writing code in a more declarative style.

Thanks to the combination of C# 3.0 and F#, this article shows the ideas in a way that should be familiar to you in C#, but also shows a further step that you can take with a primarilly functional language F#. If you're a .NET developer and you want to understand what functional programming is and how it can help you to become better and more productive then continue reading. If you'll find this article interesting, then don't forget to check out the book, which explains everything in larger detail and discusses many other interesting ideas.

Reactive Programming (IV.) - Developing a game in Reactive LINQ

In this part of the article series about Reactive LINQwe're going to implement a slightly more complicated application using the library that I introduced in the previous three articles. We're going to use basic event stream queries from the second article as well as advanced operators introduced in the third part. This time, I'll also show the F# version of all the examples, so we're going to build on the ideas from the first part.

I originally wanted to write the demo only in Visual Basic, because I think that it is really amazig to show an idea that came from functional programming in a language that no one (maybe until recently) connects with functional programming. Then I realized that I really want to show the F# version too, because F# was an inspiration for the whole Reactive LINQ idea and it is interesting alone as well. But finally, I thought that don't showing the C# version may look offensive to many readers (especially since I'm still C# MVP...). So, I ended up writing the game in all three languages, but the code is surprisingly similar in all of them!

Reactive Programming (III.) - Useful Reactive LINQ Operators

In the previous article, I introduced Reactive LINQ. I explained the different point of view that we can use when working with .NET events. The idea is that .NET events can be viewed as streams of values. The value is information about the event (such as position of a mouse click or a mouse movement). These streams can be processed using LINQ queries - we can for example filter all values that are not interesting for us using where LINQ clause. For example if we want to handle clicks only in some specified area.

In the previous article, I talked about basic LINQ query operators such as select and where and some useful methods that Reactive LINQ provides (for example for merging event streams). Today, we'll take a look at two more advanced kinds of operations that we can use for working with event streams. In particular, we'll talk about aggregation operators (that you certainly know from LINQ) and about switching. Switching is a concept from functional reactive programming and it allows us to change dynamically how the application behaves. However, I'll explain this in a more detail soon.

In this article, I'm going to use mostly C# (and some Visual Basic). The functionality that I'm describing in this part isn't part of the standard F# Event module that I discussed in the first part. I implemented most of them in F# too, but I'm not going to write the samples in both of the versions in this part. If you've seen the first two articles, you'll be definitely able to use the F# versions as well, because they follow exactly the same ideas as the C#/LINQ versions. I'm going to talk about a larger demo application in the last section and I'll show an F# version as well, so you'll see some F# examples in the next part. This part serves more as a reference of the available operators, so you may read only some parts of it, then jump to the last one (to see an exciting example!) and then come back here.

Reactive programming (II.) - Introducing Reactive LINQ

In this article I'm going to introduce my project Reactive LINQ. This is largely inspired by the ideas that come from functional reactive programming in the Haskell language and from functionality that's available for working in events in F#. I introduced these ideas in my previous article in this mini-series, so if you're interested in learning more about these interesting technologies, you should definitely read the previous article about First class events in F#.

In this article, we're going to look how to apply the same ideas to the C# language and how to use LINQ queries for processing events. This article is just another my article that shows how to implement some idea from functional programmin in C#. I believe this will once again show that the new features in the C# 3.0 aren't just about querying databases, but are more widely useful even in situations that are not directly related to data-processing.

Reactive programming (I.) - First class events in F#

I believe that the LINQ project and changes in C# 3.0 and VB 9 are interesting because they allow rewriting of many ideas from functional programming. An ability to express queries easily is one of these ideas, but it is definitely not the only one. There are many other interesting ideas. The C# 3.0 language isn't primary a functional language, so it isn't easy to discover the idea if you use only C#, but it is possible to implement it if you know the idea already.

I already wrote a few interesting C# examples that were inspired by some functional idea. I'm a big fan of the F# language, so it is not a surprise that I started with an F# version of the problem and then looked at the way to do the same thing in C#. In particular, this is how my article about building dynamic queries in C# came to the existence - the F# version used FLINQ and Quotations and then I demonstrated how to do the same in C# using expression trees. Another example is my article about asynchronous programming in C# using iterators, which shows how to implement something like F# asynchronous workflows using iterators in C# 2.0.

Functional Reactive Programming

Today, I'm going to look at another very interesting idea from functional programming. It is called Functional Reactive Programming and it comes from the Haskell community. You can find a list of related Haskell projects here. However, similar things (though they are not purely functional and simplified) are available in the F# language as well. Don Syme introduced them in his blog post called F# First Class Events: Simplicity and Compositionality in Imperative Reactive Programming. In this article, I'm going to briefly introduce the implementation available in F# and I'll extend it a little bit to allow some more interesting things. In the second article from this series, I'll show how to implement the same thing in C# 3.0 (and in VB 9 too!)

Calculating with infinite sequences on MSDN

About a year ago, I wrote an article about using lazy computations in C# 3.0. It was published by the C# Community PM Charlie Calvert at the C# Developer Center. The article was a first of two articles where I wanted to demonstrate that C# 3.0 can be used for implementing useful constructs known from functional languages. I realized that I never posted the link to the second article to my blog, so you can find the annotation and link below.

However, I remembered about these two articles because I was just working on chapters 11 and 12 of the Real-world Functional Programming in .NET book that I’m writing. Lazy values, which were the topic of my first article, are discussed in the second part of chapter 11 and IEnumerable and F# sequences are the topic for the first part of chapter 12. Because I already wrote two articles on this topic, I had to think really hard to find better (and still simple enough) examples where these concepts are useful in practice. I also finally have enough space to show how these two concepts relate and talk about some interesting background – for example in Haskell, lazy sequences are in fact just ordinary lists that are lazy thanks to the Haskell nature.

A year ago, I definitely wouldn’t believe that today, I’ll be writing about the same topics, but this time as part of a book that has partly the same goal as these two articles – to show that functional programming ideas are really useful in the real-world and can enrich your programming toolbox (no matter whether you’re using C# or F# language). Anyway, here is the link to the second article – as usual when I look at something that I worked on a long time ago, I think I should rewrite it to make it better :-), but it still gives you an idea what is the book that I’m working on about...

Functional Programming in .NET book - An update

Recently, I announced on my blog that I’m working on a book for Manning called Real world Functional Programming in .NET. The goal of the book is to explain the most interesting and useful ideas of functional programming to a real world C# developer. I'm writing this book, because I believe that functional programming is becoming increasingly important. Here is a couple of reasons why you should have this book on your bookshelf:

• Ideas behind C# 3.0 and LINQ - these main-stream technologies are inspired by functional programming and the new C# 3.0 features give us definitely much more than just a new way to query databases. The book explains the ideas behind these features and shows how to use them more efficiently.
• Learning the F# language - F# is becoming a first-class citizen in the Visual Studio family of languages, which alone would be a good reason for learning it! Even if you're not going to use it for your next large .NET project, you'll find it useful for quick prototyping of ideas and testing how .NET libraries work thanks to the great interactive tools.
• Real world examples - the book includes a large set of real-world examples that show how to develop real applications in a functional way - both in F# and C#. Among other things, the examples show how to utilize multi-core CPUs, how to better obtain and process data and how to implement animations and GUI applications in a functional way.

The book is available via the MEAP (Manning Early Access Program) and if you want to get a better idea what is the book about, you can read the first chapter for free. Anyway, it is more than a month since I posted the announcement, so I decided to write a brief update....

Announcing: Real-world Functional Programming in .NET

If you’ve been reading my blog or seen some my articles, you know that I’m a big fan of the F# language and functional programming style. I’m also often trying to present a bit different view of C# and LINQ – for me it is interesting mainly because it brings many functional features to a main-stream language and allows using of many of the functional patterns in a real-world. Elegant way for working with data, which is the most commonly used feature of C# 3.0, is just one example of this functional approach. Talking about real-world applications of functional programming, there is also fantastic news about F#. It was announced last year that F# will become fully supported Visual Studio language and the first CTP version of F# was released this week!

I always thought that the topics mentioned in the previous paragraph are really interesting and that functional programming will continue to become more and more important. That’s why I’m really excited by the news that I’d like to announce today – I’m writing a book about functional programming in F# and C#....

F# Overview (IV.) - Language Oriented Programming

In the fourth article of the F# overview series, I will shortly describe how I understad the language oriented paradigm and how the F# language can be used for developing libraries using this paradigm. We will look how discriminated unions relate to this paradigm and at three specific features that support this paradigm, namely active patterns, computation expressions and quotations.

Defining precisely what the term language oriented programming means in context of the F# language would be difficult, so I will instead explain a few examples that will demonstrate how I understand it. In general, the goal of language oriented programming is to develop a language that would be suitable for some (more specific) class of tasks and use this language for solving these tasks. Of course, developing a real programming language is extremely complex problem, so there are several ways for making it easier. As the most elementary example, you can look at XML files (with certain schema) as language that are processed by your program and solve some specific problem (for example configuring the application). As a side note, I should mention that I'm not particularly happy with the term ‘language’ in this context, because the term can be used for describing a wide range of techniques from very trivial constructs to a complex object-oriented class libraries, but I have not seen any better term for the class of techniques that I’m going to talk about.

F# Overview (III.) - Imperative and Object-Oriented Programming

In the third part of the F# Overview article series, we will look at language features that are mostly well known, because they are present in most of the currently used programming languages. Indeed, I'm talking about imperative programming, which is a common way for storing and manipulating application data and about object oriented programming which is used for structuring complex programs.

In general, F# tries to make using them together with the functional constructs described in the previous part [^] as natural as possible, which yields several very powerful language constructs.

F# Overview (II.) - Functional Programming

In the second part of the F# overview we will look at functional programming, which is probably the most important paradigm used with the F# language, because F# is built on the same grounds as many functional languages. We will first examine the standard F# data types, which are useful to allow the functional programming style and we will also look at a few functional tricks.

As already mentioned in the Introduction for this article series, F# is a typed functional language, by which I mean that types of all values are determined during the compile-time. However, thanks to the use of a type inference, the types are explicitly specified in the code very rarely as we will see in the following examples. Basic data types (aside from a standard set of primitive numeric and textual types that are present in any .NET language) available in F# are tuple, discriminated union, record, array, list, function and object. In the following quick overview, we will use the F# interactive, which is a tool that compiles and executes the entered text on the fly.

F# Overview (I.) - Introduction

In my bachelor thesis I included a short introduction that covered all of the important aspects of the F# programming language and I thought that it may be useful to extend it a little bit to cover also a topics that were not important for my thesis and post it as an article, so there is one and relatively short article that introduces all the interesting F# features. The article got however a bit longer than I expected, so I decided to split it into a three parts that would introduce three different paradigms that are supported by F#. Of course, this series won't teach you everything about F#, but it tries to cover the main F# design goals and (hopefully) presents all the features that make F# interesting and worth learning. In this first part I will shortly introduce F# and the supported paradigms that will be discussed in the upcoming articles.

Lazy Computation in C# on MSDN

I think that one of the interesting things about C# 3.0 is that it gives you the ability to use many techniques known from functional languages (like Haskell or F#). Most of the articles about C# 3.0 and LINQ focus on the queries and LINQ to SQL, but I believe that using these functional techniques deserve some attention as well. This is why I'm very happy that my article about one of these techniques - representing lazy computations - is now available at the C# Developer Center. I would like to thank to Charlie Calvert [^], who is the Community Program Manager for C# and who edited and published my article there. Here is the annotation:

Most of the programming languages used in practice (including for example C#, VB.NET, C++, Python or Java) employ so called eager evaluation, which means that the program evaluates all expression and statements in the order in which they are written, so all the preceding statements and expressions are evaluated before executing the next piece of code. This, for example, means that all arguments to a method call are evaluated before calling the method. Sometimes it may be useful to delay an execution of some code until the result is actually needed, either because the result may not be needed at all (but we can’t tell that before executing some computation) or because we don’t want to block the program for a long time by executing all computations in advance and instead we want to execute the computations later, when we will actually need the result.

In this article we will look how these lazy computations can be written in C# (using some of the new language features from version 3.0). We will first implement a Lazy class to represent this kind of computation, then look at a few simple examples to demonstrate how the class can be used, and finally we will examine one slightly more complicated, but practically useful application.

You can read the complete article here: Lazy Computation in C# [^]

Keep your multi-core CPU busy with F#

The growth of computer CPU speed is slowly being replaced by the growth of number of CPUs (or cores) in the computer at least for the close future. This causes a revolution in the way software is written, because traditional and most widely used way of writing concurrent applications using threads is difficult and brings several serious issues. Some predictions say that within a few years, almost every computer will have about 16 cores, so there is a huge need for programming paradigms or idioms that help developers write concurrent software easily (see also The Free Lunch Is Over [^] written by Herb Sutter).

Functional programming languages (especially pure functional languages) are interesting from this point of view, because the program doesn't have side-effects which makes it very easy to parallelize it (programs in pure functional languages can't have any side-effects by design, in other functional languages like F# the side-effects can be eliminated by following functional programming style).

This article describes the code that makes it possible to parallelize some common F# constructs like the List.map and List.filter...

Concepts behind the C# 3.0 language

One of the lectures that I attended last year was Programming Methodology and Philosophy of Programming Languages. The lecture was mostly about history of programming languages and how several features evolved, disappeared and than after many years appeared again in another programming language.

As I final work I decided to write an article that describes ideas that influenced the design of the C# 3.0 language. Some of these features are known from functional languages (for example from LISP or Haskell), some other were developed at Microsoft Research and appeared in the F# language or Cω. I also wanted to show in what ways are these features limited in the C# 3.0. I think that thanks to these limitation, the C# 3.0 is still a simple (or at least not difficult) to understand which is very important for mainstream language, but I find it interesting to know what is possible in other (less limited) languages.

• You can also download the article in PDF (404kB)
• The article is also available at CodeProject.com

All blog posts by tag