TP

Comparing date range handling in C# and F#

I was recently working on some code for handling date ranges in Deedle. Although Deedle is written in F#, I also wrote some internal integration code in C#. After doing that, I realized that the code I wrote is actually reusable and should be a part of Deedle itself and so I went through the process of rewriting a simple function from (fairly functional) C# to F#. This is a small (and by no means representative!) example, but I think it nicely shows some of the reasons why I like F#, so I thought I'd share it.

One thing that we are adding to Deedle is a "BigDeedle" implementation of internal data structures. The idea is that you can load very big frames and series without actually loading all data into memory.

When you perform slicing on a large series and then merge some of the parts of the series (say, years 2010, 2012 and 2014), you end up with a series that combines a couple of chunks. If you then restrict the series (say, from June 2012 to June 2014), you need to restrict the ranges of the chunks:

Demonstration

As the diagram shows, this is just a matter of iterating over the chunks, keeping those in the range, dropping those outside of the range and restrictingthe boundaries of the other chunks. So, let's start with the C# version I wrote.

Published: Wednesday, 22 April 2015, 4:55 PM
Tags: f#, c#, deedle, linq, functional programming
Read the complete article

Writing custom F# LINQ query builder

One of the attendees of my virtual F# in Finance course, Stuart recently asked me a pretty advanced question about writing custom queries with F#, because he was interested in writing a nicer querying library for Amazon DynamoDB (his project is here).

The DynamoDB could even be a type generated by a type provider (with all the tables available in Dynamo DB). Now, the above example uses the built-in query builder, which is extensible, but, as far as I know, you have to use LINQ expression trees to support it. In this article, I'm going to use an alternative approach with custom builder (so you would write dynamo { ... } instead of query { ... }).

I wanted to write a minimal example showing how to do this, so this blog post is going to be mostly code (unlike my other chatty articles!), but it should give you (and Stuart :-)) some idea how to do this. I was quite intrigued by the idea of having a nice query language for DynamoDB, so I'm hoping that this blog post can help move the project forward!

namespace Microsoft
namespace Microsoft.FSharp
namespace Microsoft.FSharp.Quotations
module Patterns

from Microsoft.FSharp.Quotations
module DerivedPatterns

from Microsoft.FSharp.Quotations
type People =
  {Name: string;
   Age: int;}

Full name: Query-translation.People
People.Name: string
Multiple items
val string : value:'T -> string

Full name: Microsoft.FSharp.Core.Operators.string

--------------------
type string = System.String

Full name: Microsoft.FSharp.Core.string
People.Age: int
Multiple items
val int : value:'T -> int (requires member op_Explicit)

Full name: Microsoft.FSharp.Core.Operators.int

--------------------
type int = int32

Full name: Microsoft.FSharp.Core.int

--------------------
type int<'Measure> = int

Full name: Microsoft.FSharp.Core.int<_>
type DynamoDB =
  static member People : seq<People>

Full name: Query-translation.DynamoDB
Multiple items
static member DynamoDB.People : seq<People>

Full name: Query-translation.DynamoDB.People

--------------------
type People =
  {Name: string;
   Age: int;}

Full name: Query-translation.People
Multiple items
val seq : sequence:seq<'T> -> seq<'T>

Full name: Microsoft.FSharp.Core.Operators.seq

--------------------
type seq<'T> = System.Collections.Generic.IEnumerable<'T>

Full name: Microsoft.FSharp.Collections.seq<_>
module Seq

from Microsoft.FSharp.Collections
val empty<'T> : seq<'T>

Full name: Microsoft.FSharp.Collections.Seq.empty
val query : Linq.QueryBuilder

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.query
val p : People
property DynamoDB.People: seq<People>
custom operation: where (bool)

Calls Linq.QueryBuilder.Where
custom operation: select ('Result)

Calls Linq.QueryBuilder.Select

Published: Tuesday, 7 April 2015, 1:41 PM
Tags: f#, functional programming, linq
Read the complete article

Processing trees with F# zipper computation

One of the less frequently advertised new features in F# 3.0 is the query syntax. It is an extension that makes it possible to add custom operations in an F# computation expression. The standard query { .. } computation uses this to define operations such as sorting (sortBy and sortByDescending) or operations for taking and skipping elements (take, takeWhile, ...). For example, you can write:

1: 
2: 
3: 
query { for x in 1 .. 10 do
        take 3
        sortByDescending x }

In this article I'll use the same notation for processing trees using the zipper pattern. I'll show how to define a computation that allows you to traverse a tree and perform transformations on (parts) of the tree. For example, we'll be able to say "Go to the left sub-tree, multiply all values by 2. Then go back and to the right sub-tree and divide all values by 2" as follows:

1: 
2: 
3: 
4: 
5: 
6: 
7: 
tree { for x in sample do
       left 
       map (x * 2) 
       up
       right
       map (x / 2) 
       top }

This example behaves quite differently to the usual query computation. It mostly relies on custom operations like left, right and up that allow us to navigate through a tree (descend along the left or right sub-tree, go back to the parent node). The only operation that does something is the map operation which transforms the current sub-tree.

This was just a brief introduction to what is possible, so let's take a detailed look at how this works...

val query : Linq.QueryBuilder

Full name: Microsoft.FSharp.Core.ExtraTopLevelOperators.query
val x : int
custom operation: take (int)

Calls Linq.QueryBuilder.Take
custom operation: sortByDescending ('Key)

Calls Linq.QueryBuilder.SortByDescending
type Tree<'T> =
  | Node of Tree<'T> * Tree<'T>
  | Leaf of 'T
  override ToString : unit -> string

Full name: Tree-zipper-query.aspx.Tree<_>
union case Tree.Node: Tree<'T> * Tree<'T> -> Tree<'T>
union case Tree.Leaf: 'T -> Tree<'T>
val x : Tree<'T>
override Tree.ToString : unit -> string

Full name: Tree-zipper-query.aspx.Tree`1.ToString
match x with
    | Node(l, r) -> sprintf "(%O, %O)" l r
    | Leaf v -> sprintf "%O" v
type Path<'T> =
  | Top
  | Left of Path<'T> * Tree<'T>
  | Right of Path<'T> * Tree<'T>
  override ToString : unit -> string

Full name: Tree-zipper-query.aspx.Path<_>
union case Path.Top: Path<'T>
union case Path.Left: Path<'T> * Tree<'T> -> Path<'T>
union case Path.Right: Path<'T> * Tree<'T> -> Path<'T>
val x : Path<'T>
override Path.ToString : unit -> string

Full name: Tree-zipper-query.aspx.Path`1.ToString
match x with
    | Top -> "T"
    | Left(p, t) -> sprintf "L(%O, %O)" p t
    | Right(p, t) -> sprintf "R(%O, %O)" p t
type TreeZipper<'T> =
  | TZ of Tree<'T> * Path<'T>
  override ToString : unit -> string

Full name: Tree-zipper-query.aspx.TreeZipper<_>
union case TreeZipper.TZ: Tree<'T> * Path<'T> -> TreeZipper<'T>
val x : TreeZipper<'T>
override TreeZipper.ToString : unit -> string

Full name: Tree-zipper-query.aspx.TreeZipper`1.ToString
let (TZ(t, p)) = x in sprintf "%O [%O]" t p
val left : _arg1:TreeZipper<'a> -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.left


 Navigates to the left sub-tree
val failwith : message:string -> 'T

Full name: Microsoft.FSharp.Core.Operators.failwith
val l : Tree<'a>
val r : Tree<'a>
val p : Path<'a>
val right : _arg1:TreeZipper<'a> -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.right


 Navigates to the right sub-tree
val current : _arg1:TreeZipper<'a> -> 'a

Full name: Tree-zipper-query.aspx.current


 Gets the value at the current position
val x : 'a
val up : _arg1:TreeZipper<'a> -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.up
val top : _arg1:TreeZipper<'a> -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.top
val t : TreeZipper<'a>
val tz : TreeZipper<'a>
Multiple items
val unit : v:'a -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.unit


 Build tree zipper with singleton tree


--------------------
type unit = Unit

Full name: Microsoft.FSharp.Core.unit
val v : 'a
val bindSub : f:('a -> TreeZipper<'a>) -> treeZip:TreeZipper<'a> -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.bindSub


 Transform leaves in the current sub-tree of 'treeZip'
 into other trees using the provided function 'f'
val f : ('a -> TreeZipper<'a>)
val treeZip : TreeZipper<'a>
val bindT : (Tree<'a> -> Tree<'a>)
val t : Tree<'a>
val current : Tree<'a>
val path : Path<'a>
Multiple items
type TreeZipperBuilder =
  new : unit -> TreeZipperBuilder
  member Current : tz:TreeZipper<'a> -> 'a
  member Current : tz:TreeZipper<'a> -> 'a
  member For : tz:TreeZipper<'T> * f:('T -> TreeZipper<'T>) -> TreeZipper<'T>
  member Left : tz:TreeZipper<'a> -> TreeZipper<'a>
  member Left : tz:TreeZipper<'a> -> TreeZipper<'a>
  member Right : tz:TreeZipper<'a> -> TreeZipper<'a>
  member Right : tz:TreeZipper<'a> -> TreeZipper<'a>
  member Select : tz:TreeZipper<'a> * f:('a -> 'a) -> TreeZipper<'a>
  member Select : tz:TreeZipper<'a> * f:('a -> 'a) -> TreeZipper<'a>
  ...

Full name: Tree-zipper-query.aspx.TreeZipperBuilder

--------------------
new : unit -> TreeZipperBuilder
val x : TreeZipperBuilder
member TreeZipperBuilder.For : tz:TreeZipper<'T> * f:('T -> TreeZipper<'T>) -> TreeZipper<'T>

Full name: Tree-zipper-query.aspx.TreeZipperBuilder.For


 Enables the 'for x in xs do ..' syntax
val tz : TreeZipper<'T>
val f : ('T -> TreeZipper<'T>)
member TreeZipperBuilder.Yield : v:'a -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.TreeZipperBuilder.Yield


 Enables the 'yield x' syntax
val tree : TreeZipperBuilder

Full name: Tree-zipper-query.aspx.tree


 Global instance of the computation builder
Multiple items
type CustomOperationAttribute =
  inherit Attribute
  new : name:string -> CustomOperationAttribute
  member AllowIntoPattern : bool
  member IsLikeGroupJoin : bool
  member IsLikeJoin : bool
  member IsLikeZip : bool
  member JoinConditionWord : string
  member MaintainsVariableSpace : bool
  member MaintainsVariableSpaceUsingBind : bool
  member Name : string
  ...

Full name: Microsoft.FSharp.Core.CustomOperationAttribute

--------------------
new : name:string -> CustomOperationAttribute
member TreeZipperBuilder.Left : tz:TreeZipper<'a> -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.TreeZipperBuilder.Left
member TreeZipperBuilder.Right : tz:TreeZipper<'a> -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.TreeZipperBuilder.Right
member TreeZipperBuilder.Up : tz:TreeZipper<'a> -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.TreeZipperBuilder.Up
member TreeZipperBuilder.Top : tz:TreeZipper<'a> -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.TreeZipperBuilder.Top
member TreeZipperBuilder.Current : tz:TreeZipper<'a> -> 'a

Full name: Tree-zipper-query.aspx.TreeZipperBuilder.Current


 Extracts the current value and returns it
member TreeZipperBuilder.Select : tz:TreeZipper<'a> * f:('a -> 'a) -> TreeZipper<'a>

Full name: Tree-zipper-query.aspx.TreeZipperBuilder.Select


 Transform the current sub-tree using 'f'
Multiple items
type ProjectionParameterAttribute =
  inherit Attribute
  new : unit -> ProjectionParameterAttribute

Full name: Microsoft.FSharp.Core.ProjectionParameterAttribute

--------------------
new : unit -> ProjectionParameterAttribute
val f : ('a -> 'a)
custom operation: left

Calls TreeZipperBuilder.Left
custom operation: map ('a)

Calls TreeZipperBuilder.Select


 Transform the current sub-tree using 'f'
custom operation: up

Calls TreeZipperBuilder.Up
custom operation: right

Calls TreeZipperBuilder.Right
custom operation: top

Calls TreeZipperBuilder.Top

Published: Wednesday, 19 December 2012, 2:22 PM
Tags: f#, haskell, research, monads, linq
Read the complete article

All blog posts by tag

f# (112), functional (66), research (44), c# (37), asynchronous (27), parallel (23), academic (22), functional programming (20), universe (20), programming languages (18), meta-programming (18), philosophy (15), links (15), presentations (14), data science (12), writing (12), joinads (12), thegamma (11), web (10), data journalism (9), math and numerics (9), random thoughts (9), talks (8), phalanger (8), haskell (7), mono (7), webcast (7), fslab (5), open source (5), visualization (4), fun (4), accelerator (4), design (3), type providers (3), linq (3), f# data (3), .net (3), training (2), coeffects (2), deedle (2), monads (2), art (2), fractals (2), funscript (2), new york (2), manning (2), books (2), teaching (1), fable (1), machine learning (1), comonads (1), fake (1), f# formatting (1), deep dives (1), async (1), events (1), trainings (1), london (1), literate (1)