Creating interactive You Draw bar chart with Compost

For a long time, I've been thinking about how to design a data visualization library that would make it easier to compose charts from simple components. On the one hand, there are charting libraries like Google Charts, which offer a long list of pre-defined charts. On the other hand, there are libraries like D3.js, which let you construct any data visualization, but in a very low-level way. There is also Vega, based the idea of grammar of graphics, which is somewhere in between, but requires you to specify charts in a fairly complex language including a huge number of transformations that you need to write in JSON.

In the spirit of functional domain specific languages, I wanted to have a small number of simple but powerful primitives that can be composed by writing code in a normal programming language like F# or JavaScript, rather than using JSON.

My final motivation for working on this was the You Draw It article series by New York Times, which uses interactive charts where the reader first has to make their own guess before seeing the actual data. I wanted to recreate this, but for bar charts, when working on visualizing government spending using The Gamma.

The code for this was somewhat hidden inside The Gamma, but last month, I finally extracted all the functionality into a new stand-alone library Compost.js with simple and clean source code on GitHub and an accompanying paper draft that describes it (PDF).

In this article, I will show how to use Compost.js to implement a "You Draw" bar chart inspired by the NYT article. When loaded, all bars show the average value. You have to drag the bars to positions that you believe represent the actual values. Once you do this, you can click "Show me how I did" and the chart will animate to show the actual data, revealing how good your guess was. Before looking at the code, you can have a look at the resulting interactive chart, showing the top 5 areas from the 2015 UK budget (in % of GDP):

Upcoming F# events - learn Suave, FsLab & more!

Some people in the F# community have reputation for traveling too much. I do not know how that is possible, but as it happens, I will be visiting a couple of places in June and doing a number of talks, workshops and courses. So, if you are thinking about getting into F#, web development with F# using the amazing Suave library, playing with the new trendy F# to JavaScript compiler called Fable, or learning about the recent features in FsLab and Ionide, then continue reading!

The map includes all my travels, but not all of the pins are for F# events. I'm visiting Prague just to see my family (even though there is a new awesome F# meetup there) and my stop in Paris is attending Symposium for the History and Philosophy of Programming (although we might still do something with the local F# group too).

Coeffects playground: Interactive essay based on my PhD thesis

In my PhD thesis, I worked on integrating contextual information into a type system of functional programming languages. For example, say your mobile application accesses something from the environment such as GPS sensor or your Facebook friends. With coeffects, this could be a part of the type. Rather than having type string -> Person, the type of a function would also include resources and would be string -{ gps, fb }-> Person. I wrote longer introduction to coeffects on this blog before.

As one might expect, the PhD thesis is more theoretical and it looks at other kinds of contextual information (e.g. past values in stream-based data-flow programming) and it identifies abstract coeffect algebra that captures the essence of contextual information that can be nicely tracked in a functional language.

I always thought that the most interesting thing about the thesis is that it gives people a nice way to think about context in a unified way. Sadly, the very theoretical presentation in the thesis makes this quite hard for those who are not doing programming language theory.

To make it a bit easier to explore the ideas behind coeffects, I wrote a coeffect playground that runs in a web browser and lets you learn about coeffects, play with two example context-aware languages, run a couple of demos and learn more about how the theory works. Go check it out now or continue below to learn more about some interesting internals!

In the age of the web: Typed functional-first programming revisited

Most programming languages were designed before the age of web. This matters because the web changes many assumptions that typed functional language designers tak for granted. For example, programs do not run in a closed world, but must instead interact with (changing and likely unreliable) services and data sources, communication is often asynchronous or event-driven, and programs need to interoperate with untyped environments like JavaScript libraries.

How can statically-typed programming languages adapt to the modern world? In this article, I look at one possible answer that is inspired by the F# language and various F# libraries. In F#, we use type providers for integration with external information sources and for integration with untyped programming environments. We use lightweight meta-programming for targeting JavaScript and computation expressions for writing asynchronous code.

This blog post is a shorter version of a ML workshop paper that I co-authored earlier this year and you should see this more as a position statement. I'm not sure if F# and the solutions shown here are the best ones, but I think they highlight very important questions in programming language design that I very much see as unsolved.

The article has two sections. First, I'll go through a simple case study showing how F# can be used to build a client-side web widget. Then, I'll discuss some of the implications for programming language design based on the example.

Comparing date range handling in C# and F#

I was recently working on some code for handling date ranges in Deedle. Although Deedle is written in F#, I also wrote some internal integration code in C#. After doing that, I realized that the code I wrote is actually reusable and should be a part of Deedle itself and so I went through the process of rewriting a simple function from (fairly functional) C# to F#. This is a small (and by no means representative!) example, but I think it nicely shows some of the reasons why I like F#, so I thought I'd share it.

One thing that we are adding to Deedle is a "BigDeedle" implementation of internal data structures. The idea is that you can load very big frames and series without actually loading all data into memory.

When you perform slicing on a large series and then merge some of the parts of the series (say, years 2010, 2012 and 2014), you end up with a series that combines a couple of chunks. If you then restrict the series (say, from June 2012 to June 2014), you need to restrict the ranges of the chunks:

As the diagram shows, this is just a matter of iterating over the chunks, keeping those in the range, dropping those outside of the range and restrictingthe boundaries of the other chunks. So, let's start with the C# version I wrote.

Writing custom F# LINQ query builder

One of the attendees of my virtual F# in Finance course, Stuart recently asked me a pretty advanced question about writing custom queries with F#, because he was interested in writing a nicer querying library for Amazon DynamoDB (his project is here).

The DynamoDB could even be a type generated by a type provider (with all the tables available in Dynamo DB). Now, the above example uses the built-in query builder, which is extensible, but, as far as I know, you have to use LINQ expression trees to support it. In this article, I'm going to use an alternative approach with custom builder (so you would write dynamo { ... } instead of query { ... }).

I wanted to write a minimal example showing how to do this, so this blog post is going to be mostly code (unlike my other chatty articles!), but it should give you (and Stuart :-)) some idea how to do this. I was quite intrigued by the idea of having a nice query language for DynamoDB, so I'm hoping that this blog post can help move the project forward!

Pattern matching in action using C# 6

On year ago, on this very day, I wrote about the open-sourcing of C# 6.0. Thanks to a very special information leak, I learned about this about a week before Microsoft officially announced it. However, my information were slightly incorrect - rather then releasing the much improved version of the language, Microsoft continued working on language version internally called "Small C#", which is now available as "C# 6" in the Visual Studio 2015 preview.

It is my understanding that, with this release, Microsoft is secretly testing the reaction of the developer audience to some of the amazing features that F# developers loved and used for the last 7 years and that are coming to C# very soon. To avoid shock, these are however carefuly hidden!

In this blog post, I'm going to show you pattern matching which is probably the most useful hidden C# feature and its improvements in C# 6. For reasons that elude me, pattern matching in C# 6 is called exception filters and has some unfortunate restrictions. But we can still use it to write nice functional code!

Upcoming F# book and event deals

Since I submitted my PhD thesis in December, I had a little bit of time to finish some of the things that I wanted to do for a really long time, but never quite found time to actually do them. This included getting the R provider to work on Mac and also creating a new web site for my various functional programming trainings and books. I even have a nice domain name:

• www.functional-programming.net

The page also discusses a couple of business reasons for looking into functional programming. So, if you're a business person wondering why you should send your developers on an F# course, the site has the answers for you too! (Or if you are developer and need a page for your boss.) The other place to check out is the official F# Software Foundation web page is another great resource and the testimonials hosted there.

The page has some information about the various trainings we're offering at fsharpWorks and about the two F# books I co-authored (Real-World Functional Programming and brand new F# Deep Dives). I'm happy that we can offer some special deals on both the books and the F# FastTrack course in London, so if you're considering getting into F#, now is a good time!

Library patterns: Why frameworks are evil

This article is a follow up to my previous blog post about functional library design, but you do not need to read the previous one, because I'll focus on a different topic.

In the previous article, I wrote about a couple of principles that I find useful when designing libraries in a functional style. This follows from my experience with building F# libraries, but the ideas are quite general and can be useful in any programming language. Previously, I wrote how multiple layers of abstraction let you build libraries that make 80% of scenarios easy while still supporting the more interesting use cases.

In this article, I'll focus on two other points from the list - how to design composable libraries and how (and why) to avoid callbacks in library design. As the title suggests, this boils down to one thing - build libraries rather than frameworks!

Library patterns: Multiple levels of abstraction

Over the last few years, I created or contributed to a number of libraries including F# Data for data access, Deedle for exploratory data science with C# and F#, Markdown parser and code-formatter F# Formatting and other fun libraries such as one for composing 3D objects used in my Christmas blog post.

Building libraries is a fun and challenging task - even if we ignore all the additional things that you need to get right such as testing, documentation and building (see my earlier blog post) and focus just on the API design. In this blog post (or perhaps a series), I'll share some of the things I learned when trying to answer the question: What should a good library look like?

I was recently watching Mark Seemann's course A Functional architecture with F#, which is a great material on designing functional applications. But I also realised that not much has been written on designing functional libraries. For some aspect, you can use functional patterns like monads (see Scott Wlaschin's presentation), but this only answers a part of the question - it tells you how to design individual types, but not an entire library...

Why you should learn F# in 2015 (and how)?

I guess that it might be a bit too late for adding to your list of new year's resulution now. But just if you still have an empty slot (or in case an originally taken slot has become available), your new year's resolution should be to get involved with F#!

Obviously, the goal of this blog post is to sell you some of my F# trainings and other materials - including the online course on F# in Finance and our FastTrack to F# course in London and New York and also the F# Deep Dives book. But to conceal this fact, I'm going to fill most of the blog post with useful information about F#, the F# Software Foundation and the F# community (but if you really just want to read about my courses, scroll down to the second section).

Composing Chrismas with F#

This blog post is a part of the awesome F# Advent Calendar (see the previous entry about writing algorithms in F# from Rick Minerich), so it inevitably needs a Christmassy theme. However, there is also going to be a serious theme for the blog post, which is domain-specific languages.

One of my favorite examples of Domain-Specific Languages is a simple OpenGL library that I wrote some time ago for composing 3D graphics in F#. You can see it in my NDC 2014 talk Domain Specific Languages, the functional way and I also used it for Solving Puzzles with F# earlier on this blog.

The nice thing about the library is that it is very simple, but is rich enough to demonstrate all the important concepts. In fact, the library is so easy to use that even 8 years old can do a talk about it. So, if you're spending Christmas with your family, perhaps you can go through this article with your children!

Welcome fsharpWorks & upcoming F# events

If you are following me or the #fsharp hashtag on Twitter, you might have already come across a link to fsharpWorks or one of the upcoming F# events organized by fsharpWorks. So, what is fsharpWorks and what are we planning for you?

Stateful computations in F# with update monads

Most discussions about monads, even in F#, start by looking at the well-known standard monads for handling state from Haskell. The reader monad gives us a way to propagate some read-only state, the writer monad makes it possible to (imperatively) produce values such as logs and the state monad encapsulates state that can be read and changed.

These are no doubt useful in Haskell, but I never found them as important for F#. The first reason is that F# supports state and mutation and often it is just easier to use a mutable state. The second reason is that you have to implement three different computation builders for them. This does not fit very well with the F# style where you need to name the computation explicitly, e.g. by writing async { ... } (See also my recent blog about the F# Computation Zoo paper).

When visiting the Tallinn university in December (thanks to James Chapman, Juhan Ernits & Tarmo Uustalu for hosting me!), I discovered the work on update monads by Danel Ahman and Tarmo Uustalu on update monads, which elegantly unifies reader, writer and state monads using a single abstraction.

In this article, I implement the idea of update monads in F#. Update monads are parameterized by acts that capture the operations that can be done on the state. This means that we can define just a single computation expression update { ... } and use it for writing computations of all three aforementioned kinds.

BREAKING: Open-source C# 6.0 released

At last, the long wait is finally over! After 4 years of waiting, the fully managed implementation of the C# compiler codenamed "Roslyn" has been finally released. In the recent months, "Roslyn" has been slowly turning into vaporware, but thanks to the recent breakthrough, the team made an enormous progress over the last two months and even implemented a number of new C# 6.0 features.

The C# 6.0 compiler, together with the full source code has been released today!

UPDATE: In case you are reading this article later than on the day when it was published, let me just point out that this was released on 1 April 2014. Just to avoid any disappointments. Have fun ;-).

Solving fun puzzles with F#

Do you need to convince your friends & family that programming can be fun? For the last Christmas, I got a puzzle as a gift. It is a number of small cubes, joined together, that can be rotated and folded to form a bigger (4x4x4) cube.

We spent the last few days of the year with family and a couple of friends and I left the puzzle on the table. Every time I walked around, someone was playing with it without much success. In the end, I said that if noone solves it until 31 December, I'll write a program to do it. Which I did between 7 PM and 8 PM and, voilà, here is what I got...

So, how do you solve a puzzle in about 1 hour on New Year's eve?

Coeffects: The next big programming challenge

Many advances in programming language design are driven by some practical motivations. Sometimes, the practical motivations are easy to see - for example, when they come from some external change such as the rise of multi-core processors. Sometimes, discovering the practical motivations is a tricky task - perhaps because everyone is used to a certain way of doing things that we do not even see how poor our current solution is.

The following two examples are related to the work done in F# (because this is what I'm the most familiar with), but you can surely find similar examples in other languages:

• Multi-core is an easy to see challenge caused by an external development. It led to the popularity of immutable data structures (and functional programming, in general) and it was also partly motivation for asynchronous workflows.

• Data access is a more subtle challenge. Technologies like LINQ make it significantly easier, but it was not easy to see that inline SQL was a poor solution. This is even more the case for F# type providers. You will not realize how poor the established data access story is until you see something like the WorldBank and R provider or CSV type provider.

I believe that the next important practical challenge for programming language designers is of the kind that is not easy to see - because we are so used to doing things in certain ways that we cannot see how poor they are. The problem is designing languages that are better at working with (and understanding) the context in which programs are executed.

The F# Computation Expression Zoo (PADL'14)

F# computation expressions are the syntactic language mechanism that is used by features like sequence expressions and asynchronous workflows. The aim of F# computation expressions is to provide a single syntactic mechanism that provides convenient notation for writing a wide range of computations.

The syntactic mechanisms that are unified by computation expressions include Haskell do notation and list comprehensions, C# iterators, asynchronous methods and LINQ queries, Scala for comprehensions and Python generators to name just a few.

Some time ago, I started working on an academic article to explain what makes computation expressions unique - and I think there is quite a few interesting aspects. Sadly, this is often not very well explained and so the general perception is more like this...

How many tuple types are there in C#?

In a recent StackOverflow question the poster asked about the difference between tupled and curried form of a function in F#. In F#, you can use pattern matching to easily define a function that takes a tuple as an argument. For example, the poster's function was a simple calculation that multiplies the number of units sold n by the price p:

1: 

let salesTuple (price, count) = price * (float count)

The function takes a single argument of type Tuple<float, int> (or, using the nicer F# notation float * int) and immediately decomposes it into two variables, price and count. The other alternative is to write a function in the curried form:

1: 

let salesCurried price count = price * (float count)

Here, we get a function of type float -> int -> float. Usually, you can read this just as a function that takes float and int and returns float. However, you can also use partial function application and call the function with just a single argument - if the price of an apple is $1.20, we can write salesCurried 1.20 to get a new function that takes just int and gives us the price of specified number of apples. The poster's question was:

So when I want to implement a function that would have taken n > 1 arguments, should I for example always use a curried function in F# (...)? Or should I take the simple route and use regular function with an n-tuple and curry later on if necessary?

You can see my answer on StackOverflow. The point of this short introduction was that the question inspired me to think about how the world looks from the C# perspective...

Power of mathematics: Reasoning about functional types

One of the most amazing aspects of mathematics is that it applies to such a wide range of areas. The same mathematical rules can be applied to completely different objects (say, forces in physics or markets in economics) and they work exactly the same way.

In this article, we'll look at one such fascinating use of mathematics - we'll use elementary school algebra to reason about functional data types.

In functional programming, the best way to start solving a problem is to think about the data types that are needed to represent the data that you will be working with. This gives you a simple starting point and a great tool to communicate and develop your ideas. I call this approach Type-First Development and I wrote about it earlier, so I won't repeat that here.

The two most elementary types in functional languages are tuples (also called pairs or product types) and discriminated unions (also called algebraic data types, case classes or sum types). It turns out that these two types are closely related to multiplication and addition in algebra...

All blog posts by tag