Blog | TomasP.Net

Using custom grouping operators in LINQ

Tomas Petricek — Sun, 07 Feb 2010 20:13:32 GMT

You can use LINQ to write queries that perform grouping of data using group by or ordering of data using orderby clause. LINQ provides the default (and the most common) implementation of both of the operations, but sometimes you may need a slightly different behavior when grouping or ordering data (this article is motivated by a question on StackOverflow which needs to do exactly that for grouping).

Let's look at a simple example, which shows when we may need a different behavior when grouping data. For example, we may have the following list of stock trades containing a name of a stock and the price of the trade (stored for example as a list of TradeInfo classes with properties Name and Price):

{ { Name = "MSFT", Price = 80.00 },
  { Name = "MSFT", Price = 70.00 },
  { Name = "GOOG", Price = 100.00 },
  { Name = "GOOG", Price = 200.00 },
  { Name = "GOOG", Price = 300.00 },
  { Name = "MSFT", Price = 30.00 },
  { Name = "MSFT", Price = 20.00 } }

Now, we may want to group adjacent trades into a single summary record which will contain the name of the stock (which is same for all trades in each group), the number of trades in the group and an average price in the group. The desired results are:

{ { Name = "MSFT", Count = 2, AvgPrice = 75.00 },
  { Name = "GOOG", Count = 3, AvgPrice = 200.00 },
  { Name = "MSFT", Count = 2, AvgPrice = 25.00 } }

The operation that we want to do is very similar to group by in LINQ, but it doesn't do quite the same thing! If we used group by, we would get only two groups as the result. However, as I wrote earlier, we want to group only adjacent trades. You could write your own extension method to do this, but then you need to leave the elegant LINQ query syntax. In this article, I'll show you how to get the desired results using a simple LINQ query with a group by clause...

Deal of the day: Real-World Functional Programming

Tomas Petricek — Sun, 24 Jan 2010 17:00:41 GMT

Some time ago, I received my copies of Real-World Functional Programming. I started working on it back in May 2008 and as many people who had more experience with writing books told me, it took longer than I was expecting! Anyway, I have to say, it was worth it, holding the actual printed book with my name on the cover is just fantastic!

The goal of the book is to present functional programming concepts and ideas in a readable form. I wanted to create a book that will teach you how to think functionally without using the usual shock therapy that people usually feel when seeing functional programming for the first time. There are already a couple of reviews that suggest I was quite successful:

Functional Programming for the Real World, by Tomas Petricek and Jon Skeet, has been a very helpful book for moving to F# from C#, as the authors do a fantastic job of helping to explain the differences between OOP and FP.
James Black at Amazon.com
This book isn’t just a simple introduction to programming in F#; it’s an introductory text on functional programming covering the many reasons why it is time for this programming paradigm to finally be accepted by mainstream programmers. And it also contains much more...
CliveT, Software Engineer at Red Gate Software
... and there are many other great comments about the book at Manning book page.

Deal of the day (January 24)

Finally, here is one great news if you're interested in getting the book! Real-World Functional Programming is Manning's Deal of the Day this Sunday, January 24. On this day, the print book is available for $20 from the Manning website, with code dotd0124.

Accelerator and F# (IV.): Composing computations with quotations

Tomas Petricek — Tue, 12 Jan 2010 03:20:36 GMT

In this article series, we're talking about the Accelerator project and I'm presenting an F# library that I implemented, which allows you to use Accelerator [references] in a more sophisticated way. We've seen two examples of using Accelerator directly (see also introduction and Game of Life). In the previous article I introduced my F# library for working with Accelerator. We've seen F# functions from the DataParallel module, we implemented an algorithm that rotates an image using these functions and finally, we've seen that we can take this ordinary F# code and run it using Accelerator. This is all possible thanks to F# quotations, which we can use to get an AST (a source code) of an F# function we wrote (if the function is marked in some special way).

In this part of the series, we're going to look at working with quotations explicitly. We'll use meta-programming techniques to work with Accelerator. Meta-programming means writing programs that manipulate with other programs or pieces of code. This is exactly what we're going to do in this article. We'll write an F# function (running on CPU) that builds a program, which we'll then run using Accelerator.

This is quite interesting approach, which isn't possible when we call Accelerator methods as standard F# functions or .NET methods. The benefit is that we'll clearly see which parts of program run on CPU and what parts execute on GPU or using X64 multi-core target. We could also perform more complicated optimizations with the code (because this wouldn't affect the readability). Just for your reference, here is the list of articles in this series in case you missed some of them:

Accelerator and F# (I.): Introduction and calculating PI
Accelerator and F# (II.): The Game of Life on GPU
Accelerator and F# (III.): Data-parallel programs using F# quotations
Accelerator and F# (IV.): Composing computations with quotations

However, enough with theory and let's take a look at some code samples! This time, we'll implement blurring of an image (also called convolution). Another example how to write this in F# using Accelerator is Satnam Singh's blog post [4]. Our example will be different, because we'll write the code as standard F# program and then have it translated to Accelerator automatically using quotations. We'll also talk about the architecture of the library that we're using and look at some performance results.

Accelerator and F# (III.): Data-parallel programs using F# quotations

Tomas Petricek — Mon, 04 Jan 2010 12:50:13 GMT

If you've been following this article series, you already know that Accelerator is a MSR library [1, 2] that allows you to run code in parallel on either multi-core CPU or using shaders on GPU (see introduction). We also discussed a direct way to use Accelerator from F# (by calling Accelerator methods directly) and implemented Conway's Game of Life. In this article, we'll look at more sophisticated way of using Accelerator from F#. We'll introduce F# quotations and look at translating 'normal' F# code to use Accelerator.

In general, F# quotations allow us to treat F# code as data structure and manipulate with it. This is very similar to C# expression trees, but the F# implementation is more powerful. We can also mark a standard method or a function with a special attribute that tells the compiler to store quotation of the body. Then we can access the quotation and traverse it or modify it. In this article we'll use a function that takes an F# quotation (containing a limited set of functions) and executes it using MSR Accelerator. Implementing this functionality is a bit complicated, so we won't discuss the implementation now. We'll leave this for some future article of this series. In future, we'll also look at other interesting possibilities that we have when writing code using quotations. Here is a list of articles in this series and of the articles that I'm planning to add:

Accelerator and F# (I.): Introduction and calculating PI
Accelerator and F# (II.): The Game of Life on GPU
Accelerator and F# (III.): Data-parallel programs using F# quotations
Accelerator and F# (IV.): Composing computations with quotations

Accelerator and F# (II.): The Game of Life on GPU

Tomas Petricek — Mon, 28 Dec 2009 21:16:03 GMT

In the previous article, I introduced the Microsoft Research Accelerator library. It allows us to write computations with arrays in C# and execute them in parallel on multi-core CPU or more interestingly, using GPU shaders. In the previous artcile, we've seen how Accelerator works and how it can be accessed from F#. In this article, we'll look at one more interesting F# demo - we'll implement the famous Conway's Game of Life [1] using Accelerator. We'll use a v2 version of Accelerator which has been announced just recently and is available from Microsoft Connect [2].

Open in new window | Double click for full-screen | Download WMV video

This article is the second one from a series about using Accelerator from F#. Today, we'll use Accelerator types directly from F# - this is the simplest possible approach and is very similar to the way you'd work with Accelerator in C#. However, we can use some nice F# features such as custom operators to make the code more readable. In the next article, we'll discuss a different approach - we'll look how to execute more "standard" F# code (that doesn't reference Accelerator explicitly) with Accelerator using F# quotations. The list of articles may change, but here is a list of articles that I'm currently planning to write:

Accelerator and F# (I.): Introduction and calculating PI
Accelerator and F# (II.): The Game of Life on GPU
Accelerator and F# (III.): Data-parallel programs using F# quotations
Accelerator and F# (IV.): Composing computations with quotations

Accelerator and F# (I.): Introduction and calculating PI

Tomas Petricek — Mon, 21 Dec 2009 03:21:03 GMT

I already wrote about two projects that I worked on during an internship at MSR back in 2007 (ASP.NET support in F# and F# WebTools). Even though this was more than 2 years ago (and I did one more internship at MSR in the meantime), I still have one more project that I never published on the web. The folks from the F# team reminded me of this project recently, so I thought I could finally publish it. The project used Microsoft Research Accelerator [1, 2], which is a C# library for developing array-based computations and executing them on a GPU. More recently, the Accelerator team at MSR published Accelerator v2 [3], which was a good motivation to update my original project...

In this article, we'll look at the simplest way of using Accelerator from F#. Accelerator provides a managed interface that can be naturally used from both C# and F#. We can use a mix of method calls and overloaded operators to describe a computation. In F#, we'll also define our additional custom operators to make the code a bit nicer. After we introduce Accelerator using a simple C# demo, we'll look how to calculate an approximate value of the PI number using a Monte-Carlo method.

This article is the first one from a series about using Accelerator from F#. The list of articles may change, but here is a list of articles that I'm currently planning to write:

Accelerator and F# (I.): Introduction and calculating PI
Accelerator and F# (II.): The Game of Life on GPU
Accelerator and F# (III.): Data-parallel programs using F# quotations
Accelerator and F# (IV.): Composing computations with quotations

Real-World Functional Programming: Completed and printed!

Tomas Petricek — Sat, 19 Dec 2009 21:54:28 GMT

If you're following my blog or if you're interested in F# or functional programming in .NET, you probably noticed that I was working on a book Real-World Functional Programming. At some point, we called it Functional Programming for the Real-World, but then we changed the title back to a better sounding version Real-World Functional Programming (subtitle With examples in F# and C#). The book is also reason for a lower number of blog posts over the last year. Over the last month or so, we were doing the final edits, reviewing the final PDF version (I fixed quite a lot minor issues, synchronized book with the Beta 2 F# release and so on). Anyway, before a few days, I received the following email (as an author, I receive the same emails as those who ordered the book through the Manning Early Access Program, so that I can see what we're sending to our dear readers):

Dear Tomas Petricek,
We are pleased to announce that Real-World Functional Programming is now complete! As a MEAP subscriber you can download your copy of the finished ebook right now! (...) This ebook is the final version, identical to the softbound edition, which is currently being printed and will be available on December 24. If you chose the printed book option when you originally subscribed, we'll ship it to you automatically—no action required from you.

Finally finished!

Yes, that's right. The book is finally completed and as far as I know, it has been printed last week! If you already ordered the book, you won't receive it before Christmas, but it should come shortly after. I can't wait to see the book actually printed. The transition from the Word drafts I initially wrote to a final PDF version was already felt fantastic and I thought "It looks like a real book!" Among other things, there are now graphical arrows with comments inside listings, which looks really great and makes code listings much easier to read. Now I can look forward to seeing the actual book. Maybe I'm too conservative, but I have to say that I'm really glad that I wrote the book before everything is going to be published just electronically!

Here is a couple of links that you may found interesting if you want to look inside the book...

Functional Programming: Available Chapter Excerpts & Discount

Tomas Petricek — Sun, 26 Jul 2009 03:41:14 GMT

The work on my book Functional Programming for the Real World is slowly getting to the end. I'm currently creating index for the last couple of chapters and doing final updates based on the feedback from reviews and also from the forum at manning.com (this means that if you have some suggestions, it's the best time to post them - I haven't yet replied to all of them, but I'll certainly do that before the manuscript will go to the production).

F# Webcast (IV.) - Developing standard .NET libraries

Tomas Petricek — Mon, 15 Jun 2009 20:09:04 GMT

In the previous parts of this webcast series we've developed an F# script that downloads RSS feeds asynchronously and in parallel and searches them for the specified keywords. We followed the usual F# development style, so after introducing the basic functional concepts, we wrote the code in the simples possible style and demonstrated how to use System.Xml and System.Net namespaces. Then we refactored the existing code, to run asynchronously and process the results potentially in parallel, which was very easy thanks to F# asynchronous workflows.

In this part of the series, we'll make the next evolutionary step of our sample application. We'll turn the code that originally used F# tuples and lists into code that uses standard .NET objects and we'll also see how to declare a class in F#. This simple modification will turn the script into an F# library that is almost indistinguishable from a library developed in C#. We'll also look how you can use the library from C# web application to show the interop between C# and F# in practice. We'll start with the code from the previous part, so if you missed that, you may want to check it out or download the source code.

F# Webcast (III.) - Using Asynchronous Workflows

Tomas Petricek — Fri, 05 Jun 2009 03:39:39 GMT

In this webcast, we'll look at improving the code for downloading and processing RSS feeds that I presented in the second part (if you didn't see earlier parts, the first one was an introduction to basic functional ideas). The previous part demonstrated how to use .NET libraries and we implemented a simple downloadUrl function for obtaining content of from the web and we've also seen how to load the data into an XML document object and how to filter items. In this part, we'll modify the code to run asynchronously and potentially in parallel. To use some of the functionality, you'll need to get FSharp.PowerPack.dll, which is available with the VS 2008 installation or as a separated download for VS 2010 [4].

Now that we have the first version of the code, we can start refactoring it. I'm using the term in a slightly vague meaning - we're of course going to change the behavior of the code. We'll wrap it into F# asynchronous workflow to run without blocking threads and we'll also run multiple downloads in parallel. However, this can still be viewed as refactoring in some sense, because we're not changing the core behavior of the code. As you can see from the webcast, these kinds of refactorings are also very nicely supported by F# syntax...

F# Webcast (II.) - Using .NET libraries

Tomas Petricek — Mon, 01 Jun 2009 14:57:53 GMT

About a week ago I posted the first part of my F# webcast series. It focused on explainining the basic ideas behind functional programming such as immutability, recursion and passing functions as arguments to other functions (or methods in C#). In the first part, we've seen some C# code to demonstrate the ideas and also a bit of F#, mainly to show the basic language features.

The second part is going to be exclusively about F#. It'll demonstrate how we can start writing a demo application that grabs data from RSS feeds and processes them. You'll learn how to access .NET libraries from F# (in particular, we'll use System.Net and System.Xml). We'll develop the code iteratively, which means that we'll start by simply enumerating the RSS elements using for loop and printing the results and then we'll refactor the code to use tuples and sequence expressions to turn it into processing code that generates a sequence of feed items. Finally we'll also demonstrate how to use some of the functions from the previous part such as List.filter in practice.

F# Webcast (I.) - Introducing functional concepts

Tomas Petricek — Mon, 25 May 2009 13:39:54 GMT

Now that Visual Studio 2010 Beta 1 is out, it is finally a good time to take a look at one of the (in my opinion) most interesting new features in the new release - the F# language. F# existed for quite a long time now as Microsoft Research project, but is now becoming a real Microsoft product. Interestingly, F# is still available as a plugin for Visual Studio 2008, so if you want to try it you don't have to install the whole new beta of 2010.

There are already many resources for learning F# including my functional programming overview, which is a Manning Greenpaper for the book Functional Programming for the Real World that I'm writing with Jon Skeet and my four-part F# introduction. There are also some useful links on the official F# web site including some talk recordings. However, I haven't yet seen any good F# webcast focusing mainly on showing F# source code, starting from simple functional concepts to the real-world features like asynchronous workflows and object-oriented programming in F#, so I decided to create one.

So, here it is...

Internship project: Reactive pattern matching

Tomas Petricek — Sun, 17 May 2009 23:00:28 GMT

I already mentioned that I was doing my second internship with Don Syme at Microsoft Research in Cambridge. This time, I was in Cambridge for 6 months from October until April, so it has been more than a month since I left, but as you can guess I didn't have time to write anything about the internship until now... There isn't much to say though, because the internship was simply fantastic. Cambridge is a beautiful place (here are some autumn and winter photos), the Microsoft Research lab in Cambridge is full of smart people, so it is a perferct working environment (except that you realize that you're not as clever as you think :-)). Also, it is just a few meters away from the Computer Laboratory of the Cambridge University, so there are always many interesting talks and seminars. So, big thanks to Don Syme, James Margetson and everyone else who I had a chance to work with during the internship.

One of the reasons why I didn't have much time to write about the internship earlier is that I was invited to the Lang.NET Symposium shortly after the end of the internship. I had a chance to talk about my project there as well and there is even a video recording from the talk (the link is below), so you can watch it to find out more about my recent F# work.

Imperative computation in F# (II.) - Writing break and continue

Tomas Petricek — Sat, 25 Apr 2009 16:31:15 GMT

As I already wrote in the first part of this series, the F# language doesn't support some of the language constructs known from imperative languages such as C#. In particular, we cannot use imperative return statement that returns the result of a function from any place in the function code. In functional languages, every construct is an expression, so to get the overall result of the function, the F# language evaluates the expression and the value of the expression is used as the result. In the previous article, we've seen that we can simulate this construct in the F# language using F# computation expressions and I showed how to implement computation named imperative that allows us to write for example the exists function for working with sequences like this:

let exists f inp = imperative {
  for v in inp do 
    if f(v) then return true
  return false }

In this article, we're going to look at two more imperative constructs and we're going to talk about break and continue. We'll see that we can quite easily extend the computation builder from the previous article to allow writing code that is syntactically very close to what you would write in C#. As I already mentioned, there are of course some performance overheads when using computation expressions, but I find it very interesting how nice imperative syntax we can get in functional F#:

imperative { 
  for x in 1 .. 10 do 
    if (x % 3 = 0) then do! continue
    printfn "number = %d" x }

The only difference between this code and the code we'd probably write if F# supported continue as a keyword is that we need to wrap the code inside the imperative computation and that we need to add the do! primitive before the continue value. Now that we've seen an example of using the continue value inside the imperative computations, let's look how we can extend the computation builder from the previous article to add this feature...

Imperative computation in F# (I.) - Returning results from a function

Tomas Petricek — Thu, 19 Mar 2009 02:05:03 GMT

One of the limitations of F# is that it doesn't very well support some of the advanced imperative language constructs such as break, continue or imperative style of returning value from a function, meaning that you can't write something like return false in the middle of the function. This has good reasons. F# doesn't in principle have the notion of currently executing statement and instead treat every code you write as an expression. Clearly, when there is no current statement, we cannot jump to other statements. If you're looking for more information about these basic principles, you can take a look at my book Real World Functional Programming, which covers this distinction in details in chapter 2, but we'll look at a brief example that will clarify this idea shortly.

Often, there is really no need to use break or other imperative constructs in F#, because you can write the same thing more elegantly using one of the provided higher order function such as Seq.exists or Seq.tryfind. However, there are still some cases where the imperative programming style makes it easier to express our original intention. Also, implementing your own higher order functions (akin to Seq.exists) would sometimes be much easier if we could just use imperative return.

So, what can be done about this?

Real World Functional Programming: Second review finished!

Tomas Petricek — Mon, 02 Mar 2009 00:40:50 GMT

I've been working on the Real World Functional Programming in .NET book for quite some time now. In fact, I had the first discussions with Michael Stephens from Manning in March last year and I started thinking about the book at that time, so it has been occupying my mind for almost a year now! Until recently, I was feeling that we're not getting much closer to the end of this project, because writing a book is just a lot of work. However, I think I can finally see that we're getting closer to actually finishing the book. At Manning, we've recently finished the second review, which means that I've just got another set of very useful comments - a big thanks to all the reviewers! I'm also getting close to finishing the first draft of the whole manuscript (depending on the reviews, the content may still change a bit, but I expect to write at most one new chapter from now). Hopefully, the drafts will soon make it to the MEAP release of the book.

Source code for Real World Functional Programming available!

Tomas Petricek — Thu, 12 Feb 2009 02:10:59 GMT

As you can see, there has been quite a bit of silence on this blog for a while. There are two reasons for that - the first is that I'm still working on the book Real World Functional Programming, so all my writing activities are fully dedicated to the book. The second reason is that I'm having a great time doing an internship in the Programming Principles and Tools group at Microsoft Research in Cambridge with the F# team and namely the F# language designer Don Syme. The photo on the left side is the entrance gate to the Trinity College of the Cambridge University taken during the few days when there was a snow. I recently started using Live Gallery, so you can find more photos from Cambridge in my online gallery. Anyway, I just wanted to post a quick update with some information (and downloads!) related to the book...

Functional Programming in .NET using C# and F# (Manning Greenpaper)

Tomas Petricek — Thu, 11 Dec 2008 01:48:07 GMT

Functional programming languages have been around for a while and were always astonishing for their ability to express the ideas in a succinct, declarative way allowing the developer to focus on the essence of problem rather than on technical details of the solution. Recently, functional paradigm is gaining new prominence as an efficient way to handle development of multi-processor, parallel and asynchronous applications.

Functional ideas are arising in C# as well as in other main-stream languages and functional languages are becoming an alternative for real-world projects. Also, Microsoft recently introduced a new language called F#, which has a strong background in traditional functional languages, but as a .NET language also benefits from the rich .NET and Visual Studio ecosystem.

Available via MEAP | 500 pages
Softbound print: March 2009 (est.)

This article is partially an excerpt from my book Real-world Functional Programming in .NET [1]. Thanks to my editors at Manning I have the permission to publish it on my blog. We’ll look at several aspects of functional programming and how the same concepts, which are essential for the functional paradigm, look in the F# and in C# 3.0 with LINQ. We will shortly look at the basic programming language features like lambda functions and type inference that are now available in both F# and C#. Functional programming isn’t only about language features, but also about using different programming style, so we’ll look at some high level concepts as well. These include using immutable data structures for developing code that can be executed in parallel and writing code in a more declarative style.

Thanks to the combination of C# 3.0 and F#, this article shows the ideas in a way that should be familiar to you in C#, but also shows a further step that you can take with a primarilly functional language F#. If you're a .NET developer and you want to understand what functional programming is and how it can help you to become better and more productive then continue reading. If you'll find this article interesting, then don't forget to check out the book, which explains everything in larger detail and discusses many other interesting ideas.

Reactive Programming (IV.) - Developing a game in Reactive LINQ

Tomas Petricek — Mon, 24 Nov 2008 03:00:00 GMT

In this part of the article series about Reactive LINQwe're going to implement a slightly more complicated application using the library that I introduced in the previous three articles. We're going to use basic event stream queries from the second article as well as advanced operators introduced in the third part. This time, I'll also show the F# version of all the examples, so we're going to build on the ideas from the first part.

I originally wanted to write the demo only in Visual Basic, because I think that it is really amazig to show an idea that came from functional programming in a language that no one (maybe until recently) connects with functional programming. Then I realized that I really want to show the F# version too, because F# was an inspiration for the whole Reactive LINQ idea and it is interesting alone as well. But finally, I thought that don't showing the C# version may look offensive to many readers (especially since I'm still C# MVP...). So, I ended up writing the game in all three languages, but the code is surprisingly similar in all of them!

Reactive Programming (III.) - Useful Reactive LINQ Operators

Tomas Petricek — Fri, 21 Nov 2008 19:59:02 GMT

In the previous article, I introduced Reactive LINQ. I explained the different point of view that we can use when working with .NET events. The idea is that .NET events can be viewed as streams of values. The value is information about the event (such as position of a mouse click or a mouse movement). These streams can be processed using LINQ queries - we can for example filter all values that are not interesting for us using where LINQ clause. For example if we want to handle clicks only in some specified area.

In the previous article, I talked about basic LINQ query operators such as select and where and some useful methods that Reactive LINQ provides (for example for merging event streams). Today, we'll take a look at two more advanced kinds of operations that we can use for working with event streams. In particular, we'll talk about aggregation operators (that you certainly know from LINQ) and about switching. Switching is a concept from functional reactive programming and it allows us to change dynamically how the application behaves. However, I'll explain this in a more detail soon.

In this article, I'm going to use mostly C# (and some Visual Basic). The functionality that I'm describing in this part isn't part of the standard F# Event module that I discussed in the first part. I implemented most of them in F# too, but I'm not going to write the samples in both of the versions in this part. If you've seen the first two articles, you'll be definitely able to use the F# versions as well, because they follow exactly the same ideas as the C#/LINQ versions. I'm going to talk about a larger demo application in the last section and I'll show an F# version as well, so you'll see some F# examples in the next part. This part serves more as a reference of the available operators, so you may read only some parts of it, then jump to the last one (to see an exciting example!) and then come back here.

Reactive programming (II.) - Introducing Reactive LINQ

Tomas Petricek — Wed, 19 Nov 2008 19:57:15 GMT

In this article I'm going to introduce my project Reactive LINQ. This is largely inspired by the ideas that come from functional reactive programming in the Haskell language and from functionality that's available for working in events in F#. I introduced these ideas in my previous article in this mini-series, so if you're interested in learning more about these interesting technologies, you should definitely read the previous article about First class events in F#.

In this article, we're going to look how to apply the same ideas to the C# language and how to use LINQ queries for processing events. This article is just another my article that shows how to implement some idea from functional programmin in C#. I believe this will once again show that the new features in the C# 3.0 aren't just about querying databases, but are more widely useful even in situations that are not directly related to data-processing.

Reactive programming (I.) - First class events in F#

Tomas Petricek — Sun, 16 Nov 2008 17:14:04 GMT

I believe that the LINQ project and changes in C# 3.0 and VB 9 are interesting because they allow rewriting of many ideas from functional programming. An ability to express queries easily is one of these ideas, but it is definitely not the only one. There are many other interesting ideas. The C# 3.0 language isn't primary a functional language, so it isn't easy to discover the idea if you use only C#, but it is possible to implement it if you know the idea already.

I already wrote a few interesting C# examples that were inspired by some functional idea. I'm a big fan of the F# language, so it is not a surprise that I started with an F# version of the problem and then looked at the way to do the same thing in C#. In particular, this is how my article about building dynamic queries in C# came to the existence - the F# version used FLINQ and Quotations and then I demonstrated how to do the same in C# using expression trees. Another example is my article about asynchronous programming in C# using iterators, which shows how to implement something like F# asynchronous workflows using iterators in C# 2.0.

Functional Reactive Programming

Today, I'm going to look at another very interesting idea from functional programming. It is called Functional Reactive Programming and it comes from the Haskell community. You can find a list of related Haskell projects here. However, similar things (though they are not purely functional and simplified) are available in the F# language as well. Don Syme introduced them in his blog post called F# First Class Events: Simplicity and Compositionality in Imperative Reactive Programming. In this article, I'm going to briefly introduce the implementation available in F# and I'll extend it a little bit to allow some more interesting things. In the second article from this series, I'll show how to implement the same thing in C# 3.0 (and in VB 9 too!)

Calculating with infinite sequences on MSDN

Tomas Petricek — Thu, 13 Nov 2008 02:36:30 GMT

About a year ago, I wrote an article about using lazy computations in C# 3.0. It was published by the C# Community PM Charlie Calvert at the C# Developer Center. The article was a first of two articles where I wanted to demonstrate that C# 3.0 can be used for implementing useful constructs known from functional languages. I realized that I never posted the link to the second article to my blog, so you can find the annotation and link below.

However, I remembered about these two articles because I was just working on chapters 11 and 12 of the Real-world Functional Programming in .NET book that I’m writing. Lazy values, which were the topic of my first article, are discussed in the second part of chapter 11 and IEnumerable and F# sequences are the topic for the first part of chapter 12. Because I already wrote two articles on this topic, I had to think really hard to find better (and still simple enough) examples where these concepts are useful in practice. I also finally have enough space to show how these two concepts relate and talk about some interesting background – for example in Haskell, lazy sequences are in fact just ordinary lists that are lazy thanks to the Haskell nature.

A year ago, I definitely wouldn’t believe that today, I’ll be writing about the same topics, but this time as part of a book that has partly the same goal as these two articles – to show that functional programming ideas are really useful in the real-world and can enrich your programming toolbox (no matter whether you’re using C# or F# language). Anyway, here is the link to the second article – as usual when I look at something that I worked on a long time ago, I think I should rewrite it to make it better :-), but it still gives you an idea what is the book that I’m working on about...

Functional Programming in .NET book - An update

Tomas Petricek — Mon, 20 Oct 2008 22:10:04 GMT

Recently, I announced on my blog that I’m working on a book for Manning called Real world Functional Programming in .NET. The goal of the book is to explain the most interesting and useful ideas of functional programming to a real world C# developer. I'm writing this book, because I believe that functional programming is becoming increasingly important. Here is a couple of reasons why you should have this book on your bookshelf:

Ideas behind C# 3.0 and LINQ - these main-stream technologies are inspired by functional programming and the new C# 3.0 features give us definitely much more than just a new way to query databases. The book explains the ideas behind these features and shows how to use them more efficiently.
Learning the F# language - F# is becoming a first-class citizen in the Visual Studio family of languages, which alone would be a good reason for learning it! Even if you're not going to use it for your next large .NET project, you'll find it useful for quick prototyping of ideas and testing how .NET libraries work thanks to the great interactive tools.
Real world examples - the book includes a large set of real-world examples that show how to develop real applications in a functional way - both in F# and C#. Among other things, the examples show how to utilize multi-core CPUs, how to better obtain and process data and how to implement animations and GUI applications in a functional way.

The book is available via the MEAP (Manning Early Access Program) and if you want to get a better idea what is the book about, you can read the first chapter for free. Anyway, it is more than a month since I posted the announcement, so I decided to write a brief update....

Announcing: Real-world Functional Programming in .NET

Tomas Petricek — Tue, 02 Sep 2008 20:03:15 GMT

If you’ve been reading my blog or seen some my articles, you know that I’m a big fan of the F# language and functional programming style. I’m also often trying to present a bit different view of C# and LINQ – for me it is interesting mainly because it brings many functional features to a main-stream language and allows using of many of the functional patterns in a real-world. Elegant way for working with data, which is the most commonly used feature of C# 3.0, is just one example of this functional approach. Talking about real-world applications of functional programming, there is also fantastic news about F#. It was announced last year that F# will become fully supported Visual Studio language and the first CTP version of F# was released this week!

I always thought that the topics mentioned in the previous paragraph are really interesting and that functional programming will continue to become more and more important. That’s why I’m really excited by the news that I’d like to announce today – I’m writing a book about functional programming in F# and C#....

Dynamic Lookup in F#

Tomas Petricek — Wed, 04 Jun 2008 01:50:08 GMT

Many people view dynamic and statically-typed languages as two distinct groups (and this is often a reason for never-ending discussions). In this article, I'll try to show one interesting example, which demonstrates that these two groups are not in fact that distinct and that you can implement a common dynamic language feature in F#, which is undoubtedly statically-typed. The feature that I'm talking about is dynamic invoke using a symbolic representation of the member (this is something that can be done using symbols in Ruby, but I'll shortly explain what exactly I mean).

I intentionally wrote statically-typed and dynamic instead of dynamically-typed. In my understanding dynamic is a broader term while dynamically-typed and statically-typed are of course two distinct groups. On the other side dynamic refers to language features that are usually available in dynamically-typed languages, just because it is easy to support them in a nice way. This doesn't mean that having a dynamic feature in a statically-typed language would be impossible - it is just more difficult to implement it in a way that would be similarly elegant.

Thesis: Client-side Scripting using Meta-programming

Tomas Petricek — Mon, 17 Mar 2008 10:07:04 GMT

I realized that I haven’t yet posted a link to my Bachelor Thesis, which I partially worked on during my visit in Microsoft Research and which I successfully defended last year. The thesis is about a client/server web framework for F# called F# WebTools, which I already mentioned here and its abstract is following:

“Ajax” programming is becoming a de-facto standard for certain types of web applications, but unfortunately developing this kind of application is a difficult task. Developers have to deal with problems like a language impedance mismatch, limited execution runtime in web browser on the client-side and no integration between client and server-side parts that are developed as a two independent applications, but typically form a single and homogenous application. In this work we present the first project that deals with all three mentioned problems but which still integrates with existing web technologies such as ASP.NET on the server and JavaScript on the client. We use the F# language for writing both client and server-side part of the web application, which makes it possible to develop client-side code in a type-safe programming language using a subset of the F# library, and we provide a way to write both server-side and client-side code as a part of single homogeneous type defining the web page logic. The code is executed heterogeneously, part as JavaScript on the client, and part as native code on the server. Finally we use monadic syntax for the separation of client and server-side code, tracking this separation through the F# type system.

The full text is available here: Client side scripting using meta-programming (PDF, 1.31MB)

New Version of Phalanger with Silverlight and VS 2008 Support

Tomas Petricek — Tue, 11 Mar 2008 01:14:21 GMT

At the Lang.NET Symposium I presented several new features in Phalanger. I already posted the presentation and samples on the blog (here) and the talk is also available as a video on the Lang.NET web site (here - actually, due to the huge demand the site is down now, but I hope it gets back soon!) Anyway, the most interesting thing is that I've used Phalanger Integration for Visual Studio 2008 during the talk, which wasn't available in any Phalanger release until now.

I have to mention one more thing before talking more about the new release - Phalanger Integration for VS 2008 works with Visual Studio Shell, which means that you can get Visual Studio 2008 with Phalanger Support for free! Yes, that's right. Microsoft offers so called Visual Studio 2008 Shell, which is an "empty" IDE with no integrated languages and you can install Phalanger Integration into this version, which means that you can get very good Phalanger IDE (for Windows) for no cost at all!

If you can't wait to install this new version, you can skip the overview article and go directly to the installation links.

F# Support for ASP.NET and Notes on Samples

Tomas Petricek — Sat, 08 Mar 2008 23:07:29 GMT

As I mentioned earlier, I spent three months as an intern in Microsoft Research in Cambridge last year and I was working with Don Syme and James Margetson from the F# team. Most of the time I was working on the F# Web Toolkit, which I introduced on the blog some time ago [1], but I also worked on a few additions that are now part of the F# release. Probably the most useful addition is a new implementation of the CodeDOM provider for the F# language which makes it possible to use ASP.NET smoothly from F# (but it can be used in some other scenarios as well) together with two ASP.NET sample applications that you can explore and use as a basis for your web sites. This was actually a part of the distribution for a few months now (I of course wanted to write this article much earlier...), so you may have already noticed, but anyway, I'd still like to write down a short description of these ASP.NET samples and also a few tips for those who're interested in writing web applications in F#.

Phalanger at Lang.NET Symposium

Tomas Petricek — Sat, 02 Feb 2008 11:51:38 GMT

Last week I attended the Lang.NET Symposium and I also did a short presentation about Phalanger (below you can find the demos and slides from my talk). By the way – during the trip from Prague to Seattle I missed a connecting flight from Paris, so I had one day to visit Paris and since I was traveling on Saturday I actually quite enjoyed it :-), so that’s where the picture comes from.

Overall it was really a fantastic event with many great talks from many interesting people. I hope that there will be recordings as a last year, so I won’t comment every topic that I found interesting (that would be really a long post). But just quickly – people from Microsoft often talked about DLR (dynamic language runtime) related topics, which was interesting as we’re planning to look at DLR in Phalanger. Luckily, Wez Furlong from the PHP community was there too, so we discussed how we could work on “PHP on DLR” project together (see also our mailing list).

There were also two talks about F# (from Luke Hoban and Harry Pierson), and it was a lot of fun to hang out with them. Finally, Erik Meijer presented the Volta project, which I was particularly interested in as it shares many goals and ideas with my F# Web Tools.

Non-Microsoft talks covered wider range of topics including Mono and Moonlight project (by Miguel de Icaza) and I actually had a chance to talk with Miguel about Phalanger and we even tried running the Helicopter sample on Moonlight – there were some issues, but it seems quite promising, so I hope to have the sample running on Moonlight on Linux quite soon! Miguel mentioned that the installation of Moonlight is currently a bit painful (due to some issues with media codecs), but it should be fixed in next few weeks, so I hope to be able to try it on my machine too!

Writing Silverlight applications in PHP

Tomas Petricek — Fri, 07 Dec 2007 17:16:14 GMT

In my last post about Phalanger I mentioned that our important goal is to support the Silverlight (2.0) platform. Shortly Silverlight is a cross-browser platform that can be used for developing client-side components that run in the web browser and contain rich media, graphics and can interactively communicate with the user. The language that can be used for writing Silverlight code can be in general any .NET language, so our goal is to allow using PHP by making Phalanger compatible with Silverlight.

First steps were already made and it is becoming possible to write some very interesting things in Silverlight using PHP, there is of course still a lot of work to do and we're discussing the future development with PHP development team (you can join the mailing list PHP on DLR for more info). In this article we will first show a very basic Silverlight example that uses PHP and later I will shortly comment more complicated example - a game (quite addicting, so be careful :-)!) where you have to fly with helicopter and avoid the walls. The source code for helicopter game is also attached, so feel free to modify it or create similar games!

If you can't wait to try the demos before looking at the sources, here are the links:

Simple car demo - Click on the car and it will move!
Helicopter game demo - Click left button to fly up!

A few notes about Phalanger future

Tomas Petricek — Sun, 02 Dec 2007 23:50:53 GMT

First, I'd like to aplogoize for the lack of new about the Phalanger project recently. I spent some time working as an intern with the F# team in MSR Cambridge and I also had final bachelor exam this autumn. Anyway, this doesn't mean that there isn't anything new and interesting regarding the Phalanger project comming soon - actually (as I believe) the opposite is true! The topics that I'm going to further discuss in this article is our upcomming support for Silverlight platform, the relation between Phalanger and DLR and also our desire to make Phalanger more open to the community, so stay tuned!

Infinite Cheese Fractal using WPF 3D and F#

Tomas Petricek — Sat, 24 Nov 2007 23:22:04 GMT

I always liked fractals, because they look like objects from another world, but on the other side if you look at some things in our world you can see many similarities with fractals (but not quite as ideal with the infinite level of precision). One of my favorite fractals is 3D version of Sierpinski carpet [1], which itself is based on very famous Cantor set. Quite long time ago I thought that it would be nice to implement animation of flying through this fractal, but I was never good in 3D graphics and it looked like a lot of work, so I never get to doing it. Luckily, now with F#, which makes it very easy to write the code to generate the fractal and with WPF 3D, which can be easily used to animate the fractal, I finally had everything I needed to do it, so here it is!

Asynchronous Programming in C# using Iterators

Tomas Petricek — Thu, 15 Nov 2007 03:08:35 GMT

In this article we will look how to write programs that perform asynchronous operations without the typical inversion of control. To briefly introduce what I mean by 'asynchronous' and 'inversion of control' - asynchronous refers to programs that perform some long running operations that don't necessary block a calling thread, for example accessing the network, calling web services or performing any other I/O operation in general. The inversion of control refers to the code structure that you have to use when writing a code that explicitly passes a C# delegate as a callback to the asynchronous method (typically called BeginSomething in .NET). The asynchronous method calls the delegate when the operation completes, which reverses the way you write the code - instead of encoding the control flow using typical language constructs (e.g. while loop) you have to use global variables and write your own control mechanism.

The funny thing about this article is that it could have been written at least 3 years ago when a beta version of Visual Studio 2005 and C# 2.0 became first available, but it is using iterators in a slightly bizarre way, so it is not easy to realize that this is possible. Actually, I will use some C# 3.0 methods in the article as well, but only extension methods and mainly just to keep the code nicer. As with my earlier article about building LINQ queries at runtime, I realized that it can be done in C# when I was playing with the F# solution (called F# Asynchronous Workflows), where this approach is very natural, so I will shortly mention the F# implementation as well.

F# Overview (IV.) - Language Oriented Programming

Tomas Petricek — Sat, 03 Nov 2007 00:00:04 GMT

In the fourth article of the F# overview series, I will shortly describe how I understad the language oriented paradigm and how the F# language can be used for developing libraries using this paradigm. We will look how discriminated unions relate to this paradigm and at three specific features that support this paradigm, namely active patterns, computation expressions and quotations.

Defining precisely what the term language oriented programming means in context of the F# language would be difficult, so I will instead explain a few examples that will demonstrate how I understand it. In general, the goal of language oriented programming is to develop a language that would be suitable for some (more specific) class of tasks and use this language for solving these tasks. Of course, developing a real programming language is extremely complex problem, so there are several ways for making it easier. As the most elementary example, you can look at XML files (with certain schema) as language that are processed by your program and solve some specific problem (for example configuring the application). As a side note, I should mention that I'm not particularly happy with the term ‘language’ in this context, because the term can be used for describing a wide range of techniques from very trivial constructs to a complex object-oriented class libraries, but I have not seen any better term for the class of techniques that I’m going to talk about.

F# Overview (III.) - Imperative and Object-Oriented Programming

Tomas Petricek — Sat, 03 Nov 2007 00:00:03 GMT

In the third part of the F# Overview article series, we will look at language features that are mostly well known, because they are present in most of the currently used programming languages. Indeed, I'm talking about imperative programming, which is a common way for storing and manipulating application data and about object oriented programming which is used for structuring complex programs.

In general, F# tries to make using them together with the functional constructs described in the previous part [^] as natural as possible, which yields several very powerful language constructs.

F# Overview (II.) - Functional Programming

Tomas Petricek — Sat, 03 Nov 2007 00:00:02 GMT

In the second part of the F# overview we will look at functional programming, which is probably the most important paradigm used with the F# language, because F# is built on the same grounds as many functional languages. We will first examine the standard F# data types, which are useful to allow the functional programming style and we will also look at a few functional tricks.

As already mentioned in the Introduction for this article series, F# is a typed functional language, by which I mean that types of all values are determined during the compile-time. However, thanks to the use of a type inference, the types are explicitly specified in the code very rarely as we will see in the following examples. Basic data types (aside from a standard set of primitive numeric and textual types that are present in any .NET language) available in F# are tuple, discriminated union, record, array, list, function and object. In the following quick overview, we will use the F# interactive, which is a tool that compiles and executes the entered text on the fly.

F# Overview (I.) - Introduction

Tomas Petricek — Sat, 03 Nov 2007 00:00:01 GMT

In my bachelor thesis I included a short introduction that covered all of the important aspects of the F# programming language and I thought that it may be useful to extend it a little bit to cover also a topics that were not important for my thesis and post it as an article, so there is one and relatively short article that introduces all the interesting F# features. The article got however a bit longer than I expected, so I decided to split it into a three parts that would introduce three different paradigms that are supported by F#. Of course, this series won't teach you everything about F#, but it tries to cover the main F# design goals and (hopefully) presents all the features that make F# interesting and worth learning. In this first part I will shortly introduce F# and the supported paradigms that will be discussed in the upcoming articles.

Lazy Computation in C# on MSDN

Tomas Petricek — Sat, 06 Oct 2007 01:29:59 GMT

I think that one of the interesting things about C# 3.0 is that it gives you the ability to use many techniques known from functional languages (like Haskell or F#). Most of the articles about C# 3.0 and LINQ focus on the queries and LINQ to SQL, but I believe that using these functional techniques deserve some attention as well. This is why I'm very happy that my article about one of these techniques - representing lazy computations - is now available at the C# Developer Center. I would like to thank to Charlie Calvert [^], who is the Community Program Manager for C# and who edited and published my article there. Here is the annotation:

Most of the programming languages used in practice (including for example C#, VB.NET, C++, Python or Java) employ so called eager evaluation, which means that the program evaluates all expression and statements in the order in which they are written, so all the preceding statements and expressions are evaluated before executing the next piece of code. This, for example, means that all arguments to a method call are evaluated before calling the method. Sometimes it may be useful to delay an execution of some code until the result is actually needed, either because the result may not be needed at all (but we can’t tell that before executing some computation) or because we don’t want to block the program for a long time by executing all computations in advance and instead we want to execute the computations later, when we will actually need the result.

In this article we will look how these lazy computations can be written in C# (using some of the new language features from version 3.0). We will first implement a Lazy class to represent this kind of computation, then look at a few simple examples to demonstrate how the class can be used, and finally we will examine one slightly more complicated, but practically useful application.

You can read the complete article here: Lazy Computation in C# [^]

F# Quotations Samples on CodePlex

Tomas Petricek — Thu, 20 Sep 2007 04:33:49 GMT

Some time ago, Granville Barnett (see his homepage and old blog [^] or a new blog [^]) had a great idea and started a CodePlex project called F# Samples [^] to host various samples written in F# to demonstrate the most important concepts of both functional programming and F#. I quite like this idea, so I asked Granville if I could join and add some samples that I wrote and today I finally found a time to update what I wanted to upload to the latest version of F# and put it online.

Building LINQ Queries at Runtime in F#

Tomas Petricek — Sat, 18 Aug 2007 02:38:11 GMT

In an article about building LINQ queries at runtime in C# 3.0, I described how you can build a LINQ query dynamically, for example by combining a set of conditions using the 'or' operator in the where clause. I mentioned that the way I implemented it is largely influenced by the F# language, which provides very natural way for manipulations with code like this. In this article I will first shortly introduce FLINQ sample, which is an F# library implementing LINQ support and than I will implement the same examples I presented in the earlier article in F#.

Building LINQ Queries at Runtime in C#

Tomas Petricek — Mon, 30 Jul 2007 02:10:17 GMT

Since the first beta versions of LINQ we could hear comments that it is perfect for queries known at compile-time, however it is not possible to use it for building queries dynamically at runtime. In this article I show that this can be actually done very well for most of the common cases. The solution offered by Microsoft (mentioned in [1]) is to build query from a string, however this has many limitations and it in fact goes completely against what LINQ tries to achieve, which is writing queries in a type-safe way with full compile-time checking. In this article I will first show a few support functions to make the life a bit easier and then we will use them for building two sample applications that allows user to build a query dynamically. The solution is largely motivated by my previous use of F#, where working with “expressions” is possible at more advanced level, however I’ll write about F# later and now let’s get back to C# 3.0...

F# Web Tools: "Ajax" applications made simple

Tomas Petricek — Fri, 13 Jul 2007 04:32:29 GMT

Traditional "Ajax" application consists of the server-side code and the client-side part written in JavaScript (the more dynamicity you want, the larger JS files you have to write), which exchanges some data with the server-side code using XmlHttpRequest, typically in JSON format. I think this approach has 3 main problems, which we tried to solve in F# Web Toolkit. There are a few projects that try to solve some of them already - the most interesting projects are Volta from Microsoft [1], Links language [3] from the University of Edinburgh and Google Web Toolkit [2], but none of the projects solve all three problems at once.

Limited client-side environment
Discontinuity between server and client side
Components in web frameworks are only server-side

The aim of the F# Web Toolkit is to solve all these three problems...

Using PHP objects from C# in a type-safe way

Tomas Petricek — Mon, 30 Apr 2007 01:26:24 GMT

When you want to call PHP scripts from mainstream .NET languages, like C# you can follow two different ways. First you can use the pure mode as I demonstrated in one of the earlier articles on PHP application called Texy!. This approach can be used only for some specific applications, because pure mode has several restrictions - the two most important restrictions are that no global code or inclusions are allowed (you have to specify all source files during the compilation), but thanks to this restrictions Phalanger is able to produce classes that are compatible with .NET and can be called from C#. Second option is to create object dynamically by its name and perform all method invocations by name too. This approach can be used with any PHP scripts, but it isn't very convenient. In this article we present new features in the Phalanger beta 4 which extend the second approach and make it possible to use objects from any PHP script in C# using type-safe way.

Phalanger 2.0 Beta 4 available!

Tomas Petricek — Sun, 22 Apr 2007 16:24:09 GMT

This week we finished new release of Phalanger (the PHP compiler for .NET platform). The goal in this release was to fix many minor bugs preventing us from running some of the famous and most often deployed open-source PHP applications, so with the new release you'll be able to run for example MediaWiki, the wiki application that evolved from system used on Wikipedia or probably the best-known PHP blogging and publishing system WordPress. Other applications that we tested can be found at Phalanger website in the list of tested PHP applications [^]. Compatibility is very important for us, so if you have troubles running any open-source PHP application, let us know and we'll include it in our list!

Another thing that we focused on is interoperability between PHP scripts compiled using Phalanger and other .NET languages. Using .NET objects from Phalanger is very intuitive and has almost no limitations, however the other way is a bit difficult because of the dynamic nature of PHP language. If your PHP application can be compiled in the Phalanger pure mode, than the PHP objects can be exported and made available to C# and other languages. The methods of objects exported using this method take object (the base class for every .NET object) as a parameter, because PHP checks types at runtime, so this may be a bit confusing for the users. In the Beta 4 we tried to target these two issues - first, the fact that you have to use pure mode and second, the fact that exported objects can't contain any information about accepted types. This is a topic for entire article, so I'll write about it soon... Stay tuned :-)!

Here are the links for Phalanger 2.0 Beta 4 release:

Phalanger 2.0 Beta 4 release page [^] - CodePlex
Phalanger MySQL extension Beta 4 [^] - CodePlex

Keep your multi-core CPU busy with F#

Tomas Petricek — Sat, 24 Mar 2007 23:13:48 GMT

The growth of computer CPU speed is slowly being replaced by the growth of number of CPUs (or cores) in the computer at least for the close future. This causes a revolution in the way software is written, because traditional and most widely used way of writing concurrent applications using threads is difficult and brings several serious issues. Some predictions say that within a few years, almost every computer will have about 16 cores, so there is a huge need for programming paradigms or idioms that help developers write concurrent software easily (see also The Free Lunch Is Over [^] written by Herb Sutter).

Functional programming languages (especially pure functional languages) are interesting from this point of view, because the program doesn't have side-effects which makes it very easy to parallelize it (programs in pure functional languages can't have any side-effects by design, in other functional languages like F# the side-effects can be eliminated by following functional programming style).

This article describes the code that makes it possible to parallelize some common F# constructs like the List.map and List.filter...

CLinq - LINQ support for the C++/CLI language

Tomas Petricek — Fri, 02 Mar 2007 17:11:05 GMT

I started working on this project, because I attended C++ class at our university and I had to do some application in C++. Because I hate doing useless projects I wanted to work on something interesting and so I started thinking whether it would be possible to enable LINQ support in C++/CLI...

C++/CLI is a very flexible language and the following example proves that enabling LINQ support in C++/CLI isn't impossible. The following database query returns name of contact and company for all customers living in London:

// create connection to database
NorthwindData db(".. connection string ..");

// declare database query
Expr<Customers^> cvar = Var<Customers^>("c");
CQuery<String^>^ q = db.QCustomers
  ->Where(clq::fun(cvar, cvar.City == "London"))
  ->Select(clq::fun(cvar, 
      cvar.ContactName + Expr<String^>(", ") + cvar.CompanyName));

// execute query and output results
for each(String^ s in q->Query)
  Console::WriteLine(s);

If you are interested in more information about CLinq project you can...

Overload resolution in Phalanger

Tomas Petricek — Thu, 15 Feb 2007 21:46:45 GMT

PHP language itself doesn't method support overloading (having two methods with same name, but different number or types of parameters). This brings an interesting problem to Phalanger, because most of .NET languages support this and if we want to be able to call any .NET object from PHP we need to add support (at least) for calling of overloaded methods. The latest Phalanger release contains overload resolution described in the Integrating PHP with CLR document [1].

For example, when calling the Console::WriteLine method (which has a lot of overloads), Phalanger dynamically generates a piece of code that we call dynamic stub, which is responsible for choosing the most appropriate overload depending on the actual parameter types. This stub is generated only once for every method, which makes this implementation very efficient. The difficult part of overload resolution is, how can the stub determine what is the best overload? PHP language has a lot of implicit conversions, so when you pass the string "10.2 Little Piggies" to a method it can be implicitly converted to float (10.2) (For more details see [2]). Another example of implicit conversion is that any boolean value can be converted to string (empty string or string "0" are converted to false, every other string is converted to true).

In this article I'll describe how does the dynamic stub look like in current version of Phalanger, what problems can it cause and how are we going to fix it in the future version!

Quotations Visualizer for F# 1.1.13.8

Tomas Petricek — Wed, 14 Feb 2007 23:35:03 GMT

Quotation Visualizer is my tool that shows visual representation of F# quotations. Stephen noticed (Thanks!) that Quotation Visualizer doesn't compile with the latest version of F# (1.1.13.8), so here is the updated version:

Download executable (51kB)
Download sources (77kB)

If you want to know more about this tool, here are links to previous articles about it:

What was wrong?

This tool is written using F# light syntax, which means that you don't have to write all the semicolons and F# compiler uses whitespace to determine end of the blocks (and some other constructs). This compiler option is relatively new so it is still evolving. First issue that I had is with class declaration - the following code is not valid with the latest F# version:

type MyForm = class inherit Form as base
  // ...
end

I used it because it looks more familiar to C# developers :-). Instead of it you have to write the following:

type MyForm = class 
  inherit Form as base
  // ...
end

The second issue was with the pipeline operator which is useful when working with lists (or other sequences). The correct syntax for using pipeline operator (for expressions that are longer than one line) is following:

let filteredList = 
       oldArray
    |> Array.to_list 
    |> List.map ( fun m -> /* ... */ ) 
    |> List.filter ( fun m -> /* ... */ )

Compiling Texy! with Phalanger

Tomas Petricek — Mon, 12 Feb 2007 00:45:45 GMT

Texy! [1] is a convertor from text format (similar to formats used in some wiki applications) to valid XHTML code written in PHP. The syntax is described at Texy! web page [2]. Unfortunately, it is only in Czech language, but the syntax is very straightforward, so you can understand it without learning Czech :-).

In this article, we'll examine how to compile Texy! using Phalanger in pure mode. In this mode it is possible to use objects from PHP like any other .NET objects, so we can later used the compiled assembly for example in the following C# code:

// Create instance of Texy! parser
Texy t = new Texy();
// Call the 'process' method and cast result to string
string parsed = (string)t.process(txtTexy.Text);
// Display parsed text using literal
ltrOutput.Text = parsed;

Can't return anonymous type from method? Really?

Tomas Petricek — Tue, 23 Jan 2007 23:54:31 GMT

One of the new features introduced in C# 3.0 which will be available in Visual Studio "Orcas" (currently in CTP version) is anonymous type. Anonymous type is something very similar to tuple type from Cω [1] (which is based on tuple types known from many functional programming languages including F#). Anonymous types are extremely useful in LINQ queries, because it allows you to construct type with several properties without declaring the type (with all the properties). Example of query with anonymous type looks like this:

var q = from c in db.Customers
          where c.Country = "Czech Republic"
          select new { FullName=c.Name+" "+c.Surname, Address=c.Address };

Ok, it's probabbly not the best example, but it demonstrates the point - you want to return some information from query and you don't need to declare type that contains FullName and Address properties before, because you need it only for this single query (and you want to return only these two fields, so you don't transfer additional data that you don't need from database).

Now let's get to the second point...

Phalanger has a new website!

Tomas Petricek — Sun, 14 Jan 2007 00:49:42 GMT

As you may know, I became new project lead for the Phalanger project [^]. Phalanger was started as a project at Charles University (where I'm studying) by Tomas Matousek, Ladislav Prosek and 4 other guys, the first version was very successful and they started wokring on the second version which introduces PHP in the family of first-class .NET languages (which makes it fully interoperable with the rest of .NET world). After releasing the second version, Tomas and Ladislav moved to Microsoft and convinced me to continue in the Phalanger development. Most of the work on the 2.0 version was already done, so we're currently working on implementing the rest of the PHP/CLR language extensions, making Phalanger more compatible with existing PHP apps and so on. For more information about the history of Phalanger visit the project history page [^].

I think that Phalanger is in the phase when it can be very interesting for both PHP and .NET communities, but it never got greater publicity and there are not very much examples and articles to demonstrate the interesting Phalanger features. To improve this we started working on the new Phalanger website where users could share their experiences, tips etc. We used PHP wiki called DokuWiki [^] (running, of course on Phalanger) and you can find the new website here:

Phalanger - Homepage [^] (the new project homepage)
Phalanger - User wiki [^] (you can edit pages in user wiki after logging in)

I also wrote two articles about Phalanger (and I'm working on more :-)) to explain the most important and interesting Phalanger features and concepts. You can find the articles at our web:

Phalanger for .NET developers [^] - focused on .NET interoperability and features interesting for C# developers
Phalanger for Mono users [^] - Phalanger 2.0 supports Mono, so here is an introduction for Mono users

I'll also continue blogging about Phalanger, so you can expect more information about Phalanger, problems that we're currently solving as well as some other tips, tricks in this blog. If you're interested only in Phalanger you can use this RSS feed to monitor new articles in this blog (it shows only articles with the Phalanger tag).

F# presentation at Czech .NET Group meeting

Tomas Petricek — Tue, 14 Nov 2006 00:52:59 GMT

On 2nd of November I did a presentation on F# and functional programming at the Czech .NET User Group meeting. Because I spent quite a lot of time with puting the presentation together I wanted to make it available to wider audience, so I translated the slides and examples to English (anyway, translating the content took me only a few minutes :-)). In case that some of the readers prefer Czech version, I attached the original documents too.

In the presentation I tried to introduce some basic concepts of functional programming (immutable values, lazy evaluation) to the audience with no experience with functional programming, as well as present some of the most interesting features of F# (like strict type system based on type inference, .NET interoperability and metaprogramming). The whole contents of the presentation is following:

Functional programming in F# - Introduction to the F# type system
Some useful functional idioms - How to do Foldl/Map/Filter functions and Lazy evaluation in C#
Interactive scripting - What is important for scripting, mathematical simulation
Interoperability between F# and other .NET languages - How to use .NET libraries from F# and F# libraries from ohter .NET languages
F# as a language for ASP.NET - How to use F# as a language for ASP.NET development
Meta-programming in F# - Meta-programming features in F# and the FLINQ project

Downloads

Presentation - Czech (889 kB), English (894 kB)
Demos - Czech (500 kB), English (501 kB)

Concepts behind the C# 3.0 language

Tomas Petricek — Sun, 15 Oct 2006 15:19:03 GMT

One of the lectures that I attended last year was Programming Methodology and Philosophy of Programming Languages. The lecture was mostly about history of programming languages and how several features evolved, disappeared and than after many years appeared again in another programming language.

As I final work I decided to write an article that describes ideas that influenced the design of the C# 3.0 language. Some of these features are known from functional languages (for example from LISP or Haskell), some other were developed at Microsoft Research and appeared in the F# language or Cω. I also wanted to show in what ways are these features limited in the C# 3.0. I think that thanks to these limitation, the C# 3.0 is still a simple (or at least not difficult) to understand which is very important for mainstream language, but I find it interesting to know what is possible in other (less limited) languages.

You can also download the article in PDF (404kB)
The article is also available at CodeProject.com

F# metaprogramming and classes

Tomas Petricek — Sat, 14 Oct 2006 01:36:10 GMT

F# quotations allows you to easily write programs that manipulate with data representation of program source code. If you're not familiar with quotations I recommend reading my previous article [1] that contains short introduction to this topic first. Quotations can be used for example for translating subset of the F# language to another code representation or another language.

To get the quotation data of the expression you can either use <@ .. @> operator or resolveTopDef function. In the first case the code written between the "<@" and "@>" is converted to data during the compilation. The resolveTopDef function allows you to get quotation data of top-level definition (function) from compiled library at runtime (you have to use --quotation-data command line switch while compiling the library). I mentioned that quotations can be used to represent only subset of the F# language. Currently, one of the quotation limitations is that it's not possible to enclose the whole class in the quotation operators. It is also not possible to get the representation of the whole class at runtime nor the representation of class members (for example methods).

In this article I'll present a simple method that makes it possible to use F# quotations for working with classes as well, however this is rather a prototype that can be used for experimenting and discovering areas where working with classes might be useful, not a fully working solution.

F# quotations visualizer - reloaded!

Tomas Petricek — Sun, 01 Oct 2006 21:39:06 GMT

Some time ago, I wrote an article about useful utility called F# quotations visualizer. This utility can be used to show visual representation of F# quotations, that can represent (subset of) source code written in F#. There are two ways that you can use to get F# quotations - first is using operators <@@ ... @@> (this returns quotation of the code written inside the operator), second method is to get quotation of top level definition from compiled F# assembly (you have to explicitly enable this using command line switch --enable-quotation-data while compiling assembly).

Because I added several new features to the original Quotations visualizer, I decided to publish the latest version - here is the list of main improvements:

Rewritten using active patterns (new F# language feature)
It is possible to extract quotations from compiled F# assembly (if it contains quotation data)
Added support for several missing language constructs

F# CodeDOM Provider on the CodePlex

Tomas Petricek — Wed, 23 Aug 2006 00:13:46 GMT

You probably already saw my post regarding CodeDOM generator for the F# language and how to use it with ASP.NET. To make it more accessible for everyone, I created project at the new Microsoft community site called CodePlex [^].

To learn more about the project, visit Project Home Page
If you want to send some comments, go to Discussion Board
You can also download latest source code - Latest Check-Ins
And finally, the latest releases can be found at the Releases Page

BTW: CodePlex looks like a really good site. It is based on Visual Studio Team System (which means that developers of the project can do most of the work directly from Visual Studio). It provides management of "Work Items" (TODO list), source control and many other useful things! For example if you have any feature requests or bug requests, send them to the discussion and I can easilly create work item from the message in the discussions.

If you are interested in this project and you want to help with developing of some parts, or if you are working on a project that is related to CodeDOM and F#, please let me know. Any help or feedback is kindly welcome!

ASP.NET web applications in F#

Tomas Petricek — Sun, 13 Aug 2006 21:06:33 GMT

CodeDOM (Code Document Object Model) is set of objects (located in System.CodeDom namespace) that can be used for representing logical structure of .NET source code. These classes are used for generating Web service references (using wsdl.exe tool), for generating typed datasets and in many other situations. The most interesting use of CodeDOM classes is in ASP.NET where ASP.NET generates code from aspx/ascx files and compiles this code into web site assemblies (together with the code written in code behind files).

This means, that you can use any language for developing ASP.NET web sites, as long as you implement CodeDOM provider that generates source code from CodeDOM structure and can compile these source files (this can be simply done by executing command line compiler). I was recently working on CodeProvider for the F# language, and finally it supports everything what is needed by ASP.NET (however it is complete and it doesn't work for example with wsdl.exe). Using this CodeDOM provider you can write ENTIRE web site in F# (including in-line code enclosed in <% ... source code ... %>). I also created project template for F# web site that can be imported to Visual Studio 2005, so you can easilly try writing web pages in F#...

LINQ extensions - Simplified keyword search

Tomas Petricek — Fri, 28 Jul 2006 03:57:38 GMT

Recently, I came across interesting question at LINQ Forums (Dynamic conditions: How to achieve multiple "OR" conditions with LINQ? [1]). The question is whether LINQ (and especially LINQ to SQL) provides any simple way to return only records that contain one or more of specified keywords in the name. The question looks simple, but it is simple only if you know the number of keywords that you want to look for. In this case you can write following LINQ query:

// Products that contain "kwd1" or "kwd2" in the name 
var q = from p in db.Products
    where p.ProductName.Contains("kwd1") || p.ProductName.Contains("kwd2")
    select p;

The problem with previous code is that you can't use it if the list of keywords is dynamically entered by user (and so its length may vary). Of course, if you want to run query on in-memory data, you can get very nice results by writing extension method called ContainsAny that performs test for keyword array, but if you want to be able to translate query to SQL, the situation is a bit complicated.

F# quotations visualizer

Tomas Petricek — Wed, 21 Jun 2006 02:20:03 GMT

I already explained what F# quotations are and I explained how you can do some simple manipulations with it. In this article I'd like to present an application that I wrote and that can be helpful when working with quotations. It displays clear graphical representation of given F# quotation (using Windows Forms TreeView control).

Calling functions in LINQ queries

Tomas Petricek — Sat, 10 Jun 2006 14:26:52 GMT

The LINQ Project [^] is an extension to .NET Framework and most important .NET languages (C# and VB.Net) that extends these languages with query operators and some language features that make it possible to integrate queries in the languages. Thanks to LINQ you can write queries that read data from database (or any other data source). For example, imagine that you want to write set of queries for eshop and you need to perform a price calculation in more queries. The problem with LINQ queries is that you can't simply call a function written in C# that calculates price. The following example is NOT WORKING for this reason:

// function used in filter
static decimal CalcPrice(Nwind.Product p)
{
  return p.UnitPrice * 1.19m;
}

// query that uses MyFunc
var q = 
  from p in db.Products
  where CalcPrice(p) > 30m
  select p

I think that this is a big limitation, because when you want to keep some more complex logic in the data access layer you should be able to reuse parts of queries that are similar across more queries. The good thing is that with latest release, LINQ became extensible so it is possible to write a extensions that allow this scenario...

You can also download the article in PDF (80kB)

Slides and demos from F# presentation

Tomas Petricek — Fri, 09 Jun 2006 21:14:39 GMT

This semester I attended Advanced .NET Seminar that was led by Tomas Matousek [^] who is one of the authors of Phalanger project [^] (Which is an amazing project by the way. It takes PHP source code and compiles it without any modification to .NET). Seminar was mostly focused on Rotor and .NET internals, so if you want to learn more about these topics you can look at Advanced .NET programming [^] slides (by Tomas Matousek).

I did one presentation at this seminar too. It was about the F# language developed at Microsoft Research. It was just a quick overview of F# features, because F# is very rich topic, so it coveres only the language (functional vs. imperative behavior), F# type system, compilation of F# constructs to .NET and interoperability with .NET (for example how to create windows forms application in F#). At the end, I also mentioned F# meta-programming that allows you to look at F# code as data.

The F# language (67kB) - presentation slides in PDF
F# Samples (54kB) - zipped VS 2005 solution with samples

F# - Simple quotations transofration

Tomas Petricek — Sun, 28 May 2006 15:58:20 GMT

This article describes very simple code that I wrote while learning how to work with the F# quotations library. Using the F# quotations you can get tree representation of the quoted expression. This allows you to write code that takes code written in F# as data and performs some code analysis or compiles/translates that code to different language. This is very similar to the new C# 3.0 expression trees where you can get expression tree from lambda expression and translate this tree for example to SQL (using DLINQ). However expression trees in C# 3.0 are very limited when compared with F# quotations, so that's one of the many reasons why F# is interesting language.

Aho-Corasick string matching in C#

Tomas Petricek — Sun, 04 Dec 2005 00:18:30 GMT

I implemented this algorithm because I worked on one project where we needed to filter bad language in comments submited by users (You wouldn't believe what anonymous users sometimes write). First I tried simple solution using String.IndexOf and using Regex, but none of these solutions was very suitable for this problem, so I decided to implement Aho-Corasick algorithm which is probabbly the best algorithm for this purpose.

Article (published here an on CodeProject.com) describes implementation of this algorithm for pattern matching. In simple words this algorithm can be used for searching text for specified keywords. This implementation is usefull when you have a set of keywords and you want to find all occurences in text or check if any of the keywords is present in the text. You should use this algorithm especially if you have large number of keywords that don't change often, because in this case it is much more efficient than other algorithms that can be simply implemented using .NET class library.

Aho-Corasick search algorithm is very efficient if you want to find large number of keywords in the text, but if you want to search only for a few keywords it is better to use simple method using String.IndexOf.