Updated: 16th May 2010 after some feedback

Take these two classes for example:

    public class User
    {
        public string Id { get; set; }
        public string Name { get; set; }
        public IList<Role> Roles { get; set; }
    }

    public class Role
    {
        public string Name { get; set; }
        public int AccessLevel { get; set; }
    }

and given the following data:

            var standardRole = new Role {Name = "Standard", AccessLevel = 2};
            var adminRole = new Role { Name = "Standard", AccessLevel = 5 };
            var user1 = new User()
            {
                Id = "jbloggs",
                Name = "Joe Bloggs",
                Roles = new List<Role>
                            {
                                standardRole,
                                adminRole
                            }
            };

            var user2 = new User()
            {
                Id = "tsmith",
                Name = "Tony Smith",
                Roles = new List<Role>
                            {
                                standardRole,
                            }
            };

Now say I want to find all users with access level greater than or equal to 5. In Linq-2-sql or plain sql your might write this query like the following.

var query = list.Where(x => x.Roles.Where(y => y.AccessLevel >= 5).Count() > 0).ToList();

This query will not work in the NoRM linq provider, however there is an alternative.

1. var queryforMongo = list.Where(x => x.Roles[0].AccessLevel >= 5).ToList();or

2. var queryforMongo = list.Where(x => x.Roles.Any(y=>y.AccessLevel >= 5)).ToList();

Query 1 here will only return Users where the first role has an access level greater than or equal to 5.

Query 2 will return Users where 1 of the Roles has an access level greater than or equal to 5.

One caveat though. If you try and do a complex query ( one involving an “or” condition or a “and” using the same property ) then you will get an exception. This is because complex queries get converted to a javascript function that cannot be used in the same ways.

Hope this comes in handy for folks out there.

Adam

It currently supports a lot of functionality including deep queries, regex, datetime which is really exciting. In this post though I am going to concentrate on regular expressions.

This is the newest part of the Linq provider however it is probably the most powerful, especially for complex queries. The reason for this is that MongoDB will use the indexes created when a regular expression is used (where the query is not a complex query). A complex query is one that filters on the same property twice, uses a string function (replace/substring/toLower etc) or does some other fancy stuff. For example using the toUpper() method.

var products = session.Products.Where(x => x.Name.ToUpper() == "TEST3").ToList();

Anyways….i digress. So, here is an example of using a Regex in a Linq Query. Pretty simple.

var products = session.Products.Where(p => Regex.IsMatch(p.Name, "^te")).ToList();

Using the static Regex.IsMatch is the only way to invoke a regex call using the Linq Provider. This will however run blazingly fast. I tested this query on 1,000,000 Products and it only took 1.5sec, which was approximately 10x faster than when a complex query is invoked. There are however 3 string functions that have been optimized using this regex functionality. They are StartsWith(), EndsWith() and Contains() which is why the following query only takes 2secs to return over 48,000 rows, however when using Skip() and Take() you can get 50 results back in just milliseconds.

var products = session.Products.Where(x => x.Name.StartsWith("X")).ToList();

The Linq provider also supports 3 of the RegexOptions. They are RegexOptions.IgnoreCase (but please note this will not use the index so will be slower), RegexOptions.Multiline and RegexOptions.None. Regex’s can also be used in conjunction with other filters. eg.

var products = session.Products.Where(p => Regex.IsMatch(p.Name, "^te") && p.Price == 10).ToList();

This query is not considered a complex query because two different properties are used and an “and”(&&) operator is used.

Please note: Any time a “or” (||) operator is used, it will be considered a complex query.

Summary:

If you have a filter than can be written as a regex, chances are it will be as fast or faster than without using a regex.

So please go and try out NoRM and enjoy the freedom. I will try and post some more cool stuff in the LINQ provider over the next few weeks. Stay tuned.

        var reference_number_old = "$##%@";
        while (recordsExist)
        {
            reference_number = String(rs(0));

            if (reference_number_old != reference_number)
            {
                Response.Write(reference_number);
                reference_number_old = reference_number;
            }

            //Do more stuff
        }

Sooo very very painful, and that’s only grouping by one column (reference number).

Now however with OO programming and the advance of Linq there is a much nicer and easier way to deal with this Master-Detail type one-to-many relationship.

Lets see how we do it now, still using our own custom sql.

Firstly here is our class to represent our flat data straight out of the database.

        public class FlatCustomerAndOrders
        {
            public int ReferenceNumber { get; set; }
            public string Name { get; set; }
            public string Address { get; set; }
            public int OrderId { get; set; }
            public string ProductName { get; set; }
            public string Description { get; set; }
            public decimal Amount { get; set; }
        }

Even filling this class used to be a bit painful before Linq. You would have to create a connection, create a reader, then instantiate the FlatCustomerAndOrders class. Set the properties then add the object to a list while records could still be read. Here’s how I do it now.

        DataContext dc = new DataContext(new SqlConnection(connString));
        IEnumerable<FlatCustomerAndOrders> flat =
            dc.ExecuteQuery<FlatCustomerAndOrders>("select c.referencenumber, c.name," +
            "c.address, o.orderid, o.productname, o.description, o.amount from customers " +
            "c inner join orders o on c.referencenumber = o.referencenumber");

Thats its. We have our List of FlatCustomerAndOrders.

The next step is to organise the data so that the data is nested so its easy to display with just two foreach loops. The two classes:

        public class Customer
        {
            public int ReferenceNumber { get; set; }
            public string Name { get; set; }
            public string Address { get; set; }
            public List<Order> Orders { get; set; }
        }

        public class Order
        {
            public int OrderId { get; set; }
            public string ProductName { get; set; }
            public string Description { get; set; }
            public decimal Amount { get; set; }
        }

To fill these two class we do some clever Linq using the GroupBy method. Here’s the code.

        IList<Customer> Customers =
            flat.GroupBy(cust => new { cust.ReferenceNumber, cust.Name, cust.Address })
                .Select(c => new Customer()
                {
                    ReferenceNumber = c.Key.ReferenceNumber,
                    Name = c.Key.Name,
                    Address = c.Key.Address,
                    Orders = c.Select(o => new Order()
                    {
                        OrderId = o.OrderId,
                        ProductName = o.ProductName,
                        Description = o.Description,
                        Amount = o.Amount
                    }).ToList()
                }).ToList();

And that’s it. It may look like a bit of a mouthful but its quite simple really.

It just takes the flat list and applies the groupby method to it grouping by the three customer columns. It then builds a new list to fill the Orders property with.

Its then super easy to display the data like so.

        foreach (Customer c in Customers)
        {
            //Display the customer details

            foreach (Order o in c.Orders)
            {
                //Display the orders for each customer
            }
        }

Well I hope this helps you as much as it has helped me.

Cheers

Schotime

There are many situations where you need to read data from a file into memory or a database. Lets consider this scenario.

We have a tab delimited data file that has been exported from some web or windows application etc. Below is the sample file.

carrots    4    0.64
beans      3    0.12
oranges    7    1.02
apples     5    0.87

This file includes three columns. Column 1is the name of the product. Column 2 is the quantity and Column 3 is the unit price. We would also like a total price column derived from column 2 being multiplied by column 3.

Our aim here is to load these into our application programmatically and then bind them to a gridview. Normally you would have to read the file in line by line then do a split on each line, enter each line into a prebuilt class or datatable then add a new column and multiply the quantity by the price to obtain the total. However by using a nice little extension helper method, all of this can be rolled into just a few lines of code.

public static IEnumerable<string> ReadLinesFromFile(string filename)
{
    using (StreamReader reader = new StreamReader(filename))
    {
        while (true)
        {
            string s = reader.ReadLine();
            if (s == null)
                break;
            yield return s;
        }
    }
}

Above is the extension method to convert lines of a file into a IEnumerable list of strings.

var products = from line in ReadLinesFromFile(@"c:\import.txt")
               let item = line.Split('\t')
               select new
               {
                   Product = item[0],
                   Quantity = Convert.ToInt32(item[1]),
                   Price = Convert.ToDecimal(item[2]),
                   Total = Convert.ToInt32(item[1]) * Convert.ToDecimal(item[2])
               };

GridView1.DataSource = products;
GridView1.DataBind();

What this gives us is a list of products, with each one having 4 properties associated with it. We can then bind this straight to a gridview by setting the datasource of the gridview to products and calling the DataBind() method. This will display a table with 4 columns with the headers Product, Quantity, Price and Total.

The Linq statement can then be modified in many ways to accommodate different file formats (eg. CSV or fixed width) with a few simple changes.

This is just another way Linq is making things easier!

Schotime

One powerful feature of programing is Regular Expressions. These provide a concise and flexible means for identifying text of interest, such as particular characters, words, or patterns of characters. So whilst going over some old code of mine to extract data from a remote website, I decided to give the Regular Expression part of my code a face lift with Linq.

The code below is the setup code just to give some background.

String StringToMatch = "<tr class=\"ar1\"><td>456642</td>"
        + "<td class=\"left\">John</td>"
        + "<td class=\"left\">Smith</td>"
        + "<td>j.smith@email.com</td></tr>"
        + "<tr class=\"ar1\"><td>456643</td>"
        + "<td class=\"left\">Edward</td>"
        + "<td class=\"left\">Norman</td>"
        + "<td>e.norman@email.com</td></tr>";

Regex r = new Regex("<tr class=\"(?:ar1|ar2)\"><td>([0-9]+)</td>"
        + "<td class=\"left\">(.*?)</td>"
        + "<td class=\"left\">(.*?)</td>"
        + "<td>(.*?)</td></tr>");

MatchCollection matches = r.Matches(StringToMatch);

The following code is the preLinq version of the code to process the Regular Expression.

List<Profile> Profiles = new List<Profile>();

if (matches.Count > 0)
{
    foreach (Match m in matches)
    {
        Profile p = new Profile();

        p.Id = m.Groups[1].Value;
        p.Firstname = m.Groups[2].Value;
        p.Lastname = m.Groups[3].Value;
        p.Email = m.Groups[4].Value;

        Profiles.Add(p);
    }
}

As you can see above, a strongly typed List of type Profile is created. Then we loop through each match, first creating a new instance of the Profile object. Filling the object up with the results from our Regular Expression and finally adding it to the list. Whilst this code is pretty straight forward, look how easily Linq handles this scenario.

if (matches.Count > 0)
{
     List<Profile> Profiles = (from Match m in matches
                              select new Profile
                              {
                                  Id = m.Groups[1].Value,
                                  Firstname = m.Groups[2].Value,
                                  Lastname = m.Groups[3].Value,
                                  Email = m.Groups[4].Value
                              }).ToList();
}

As you can see above, we have managed to reduced the amount of statements from around 8 to 1. So what this is doing in english is creating a strongly typed List of the type Profile and using Linq to fill it. It states that for ever Match m in the list matches, create a new object Profile and auto initialise the variables with the values contained in the match. Finally we convert the IEnumberable<Profile> result to a List<Profile> by using the method ToList().

How easy was that! Now say you wanted the list of Profile’s sorted by lastname. Well you would normally have to build the list as above and then call the Sort method using a defined Comparison object. This is where Linq becomes even more powerful. Simply by adding one line to the Linq statement above, the List generated will be sorted by lastname.

if (matches.Count > 0)
{
     List<Profile> Profiles = (from Match m in matches
                       --->   orderby m.Groups[3].Value
                              select new Profile
                              {
                                  Id = m.Groups[1].Value,
                                  Firstname = m.Groups[2].Value,
                                  Lastname = m.Groups[3].Value,
                                  Email = m.Groups[4].Value
                              }).ToList();
}

Also another point to add regarding the definition of class Profile. Back in .NET 2.0 days creating a class was pretty painful. A lot of repeated code just to get and object with some variables.

public class Profile
{
    private string _id;
    private string _firstname;
    private string _lastname;
    private string _email;

    public string Id
    {
        get { return _id; }
        set { _id = value; }
    }

    public string Firstname
    {
        get { return _firstname; }
        set { _firstname = value; }
    }

    public string Lastname
    {
        get { return _lastname; }
        set { _lastname = value; }
    }

    public string Email
    {
        get { return _email; }
        set { _email = value; }
    }
}

As you can see its way to long. Lets see how post .NET 2.0 does it.

public class Profile
{
    public string Id { get; set; }
    public string Firstname { get; set; }
    public string Lastname { get; set; }
    public string Email { get; set; }
}

Now thats what i’m talking about. Good work team. Thats how easy it should be to create a class!

Til’ Next Time, It’s Schotime Out!