Category Archives: .NET
A Glimpse into PetaPoco
Last week I blogged about upcoming performance statistics that enable you to see all the Sql’s that were executed by PetaPoco in the current request.
It was also just over a week ago that I learned about Glimpse from this Mix video by Scott Hanselman. And wow what an impact. Anthony van der Hoorn (@anthony_vdh) and Nik Molnar (@nikmd23) had created a wonderful diagnostics system for Asp.net MVC. Firebug for the server if you will.
I then wondered if I could somehow plug PetaPoco diagnostics into the display for Glimpse and it was unbelievably easy. Implementing a single interface was all it took. Less than 50 lines of code in total to produce the following results. I will show you how to create your own tab in a future post. Update: Creating Your Own Glimpse Plugin
This gives a pretty clear display of exactly what queries were performed during the current request.
Another powerful feature of Glimpse is that it saves the last n (default 5) requests so it is indeed possible to get access to the Sql’s performed when posting data back to the server.
It really is a credit to the guys for a wonderful project which still hasn’t reached a V1.
My aim is to have both the integration to Glimpse and html source statistics available within the week but it all depends on whether the integration points make it into the main branch before hand.
Update (01/06/2011): This is now available as a package from NuGet called PetaPoco.Glimpse.
Until then, check out Glimpse (on NuGet as well) and support it by providing valuable feedback to the team.
Adam
PetaPoco – Why I’m Using A Micro-ORM
As I mentioned a few posts ago, in the last couple of years I moved away from writing native SQL to a full blown ORM and back again. Well not all the way back. Here is where PetaPoco comes in.
I had been using my own SQL to Object mapper for a while before I stumbled over PetaPoco, and it had very similar ideas to mine. It has gone one step further though, with nice support for paging, inserts, updates, deletes and multiple database support.
Since forking the repository at Github, I have implemented and suggested many changes that have made it back (thanks to Brad) into the main repository. Some of these include support for Oracle (including sequences for PK), DB field to Object field name conventions (and the ability to customize these per project).
So why have I come the full circle on this.
- Well firstly, SQL is a great query language for RDMS databases. It is very very powerful and I don’t think that it is all that difficult to learn.
- Just give me my data. I just want to create a sql query and map it to my object so I can use it. I don’t want to have explicit mapping files/code.
- The pain is just not worth it IMO. Why should I have to spend excess time trying to figure out if this query is going to produce a select n+1 or that I have to traverse 4 deep to get a piece of data?
- One file. Its not 1 DLL or 10 DLLs. Its one cs file that you just drop in to your project. It doesn’t get much simpler than that.
- Performance. I just don’t have to worry that I’m losing any performance between the database and my object. If there are performance issues, its either the query or the business logic.
And those are just some of the reasons.
To see how to use the basic features of PetaPoco, please visit the PetaPoco home page.
At the moment my fork contains a few extra features that are not currently in Brad’s repository. These include multiple primary key support for legacy database, concurrency using a version column, an OnCommandExecuted extension point and a IDatabase interface. I also have the PetaPoco.cs file set to “No Dynamic” at the moment as I’m using it with .NET 3.5. To enable the dynamic object mapping support, comment out the follow line (currently line 11).
#define PETAPOCO_NO_DYNAMIC
In future blog posts I intend to go through a few examples of how I have using it and how the extension points enable you to do some pretty interesting things.
Now I know this won’t be for everyone but unless forced, I won’t be going back to a full blown ORM.
Adam
PetaPoco Performance Statistics
The other day Sam from stackoverflow blogged about a slow page caused by a combination of problems resulting in them using Dapper to map native SQL to the objects resulting in a nice performance improvement.
I liked how he made the queries executed for the current request display in html comments in the html source, so I thought I would see if I could implement the same thing with PetaPoco. I intend to go through how I implemented the following in future posts (after I go through some basics), but for the moment I’ll give you a quick screen grab of what it looks like. Thanks goes to Sam for providing the inspiration.
Increasing Event Binding Performance using Jquery Delegate
Today I was presented with a problem when binding click events to action icons. You know the scenario; you have a table with a list of data and one of the columns has icons/links like “edit”, “new”, “view”, “delete” etc. This is how I’m using the jQuery to bind the click event to the links like so.
$("a.action").click(function() { // perform tasks });
This worked great, however when the list grew to 50 or more items, IE (yes i know) started complaining that the script is running too long. Now at the moment I have 7 actions multiplied by 50 items in the list which means it was looping 350 times. It shouldn’t be that bad but before IE9 JS performance is woeful.
To fix this issue, I needed to reduce the amount of event bindings. This is where jQuery.delegate() comes in. The syntax is a little different but its pretty easy to understand.
$("#listtable").delegate("a.action", "click", function() { // perform tasks });
The selector in this case is the table where the links are contained within (children). The first parameter is the filter for the link that needs to be clicked. The second parameter is the event name. Custom events can be used here. And the third parameter is the callback function. So its pretty similar, however instead of creating 50 events, one for each row, I’m binding the event to the surrounding table element and then using the fact that the events bubble up through parent elements to catch the appropriate event.
IE happy, Adam happy!
Getting Back In Touch With SQL
I’ve always used native SQL for my applications. Approximately a year or so ago I finally decided to give NHibernate a look and see what all the hype was about. I gave it a test run on a new project as there is really nothing like real world requirements to show you what a tool is really made of.
It started off slowly, but was really enjoying the easy of which you could do inserts and updates. Even selects where pretty easy. At least at first.
As the system started to grow, the pain started. The queries were relatively straight forward, but the object graph needed to pull all the data together for a simple nested list was becoming convoluted to say the least. At once stage I think I spend almost a day trying to figure out a query, however at the time I was still learning so I knew there would be some pain but it was starting to become ridiculous.
Let me say though, I’m not entirely blaming NHibernate (or ORMS eg. EF) for the problems. The data interactions that I was trying to perform were pretty intense, with complex hierarchical relationships, but I knew there had to be a better way.
I started to switch most of the queries over to using session.CreateSqlQuery(sql) and using the AliasToBeanTransformer to project the query into a DTO that more closely represented how the data was to be displayed on the page.
It was at this time that CQRS was starting to gain some traction and it was all starting to make a lot of sense.
How you model the data is usually not how you need to display the data.
I couldn’t believe how simple everything was after separating the queries from the commands. It was definitely an “aha” moment!
I am currently not implementing CQRS in full (event sourcing, table per model, separate read store etc) but I think it is a great compromise.
I have started to put together a sample application of how I am doing Web Applications using CQRS-Lite and I will endeavour to blog about the details in the upcoming weeks. This will also include details on the new Micro-ORM PetaPoco that I have been helping Brad out on, and how much easier it has made things!
Adam
Dynamic Dot Less Css With Asp.net MVC 2
I have been having a look at the best way to theme a asp.net mvc website in the last few weeks. I heard about dot less css late last year but hadn’t had time to integrate it into a project until now.
Integrating dot less css is pretty easy by default, as shown on its home page however I wanted something that could be configured by application users. We want to be able to specify the link html tag like the following.
<link href="/public/css/sites.less" rel="stylesheet" type="text/css" /> |
There are few steps involved in the solution I have come up with so bare with me as I go through them. Firstly we’ll start with the web.config:
<compilation> <buildProviders> <add extension=".lessx" type="System.Web.Compilation.PageBuildProvider" /> </buildProviders> </compilation> |
Insert the builderProviders block inside the compilation section. The compilation section will be under the system.web section. This will enable us to use code blocks in our lessx files.
Secondly, is the routing.
//Special Route for css so that images are relative to it routes.MapRoute("css", "public/css/{filename}.less", new { controller = "Css", action = "Index” }, new[] { "eLearning.Controllers" }); |
This specifies that any url matching /public/css/{filename}.less should be routed to the CssController passing the filename as the parameter to the Index method.
public class CssController : BaseController { [OutputCache(Duration = 10, VaryByParam = "")] public ActionResult Index(string filename) { if (string.IsNullOrEmpty(filename)) throw new ArgumentException("A filename must be supplied"); ViewData["baseColor"] = "#aabbcc"; var less = RenderViewToString(filename + ".lessx", null); var css = LessCss.Generate(less, true); return Content(css, "text/css"); } } public static class LessCss { public static string Generate(string less, bool minify) { var lessEngine = new EngineFactory(); lessEngine.Configuration.MinifyOutput = minify; var output = lessEngine.GetEngine().TransformToCss(less, null); return output; } } |
This is the css controller class. It receives the filename as the input, finds the corresponding .lessx file and returns the file with appropriate view data replaced. Is then takes the less formatted string and runs it through the dot less parser to return a css string. This is then returned to the browser with the appropriate content-type. One method I have not shown is the RenderViewToString() and how the .lessx file is located. This method is located on the base controller class that the css controller inherits from. I have also turned Caching on so that it will cache the result for 10seconds.
protected string RenderViewToString(string viewName, object model) { if (string.IsNullOrEmpty(viewName)) viewName = ControllerContext.RouteData.GetRequiredString("action"); ViewData.Model = model; using (StringWriter sw = new StringWriter()) { ViewEngineResult viewResult = ViewEngines.Engines.FindView(ControllerContext, viewName, null); ViewContext viewContext = new ViewContext(ControllerContext, viewResult.View, ViewData, TempData, sw); viewResult.View.Render(viewContext, sw); return sw.GetStringBuilder().ToString(); } } |
This method finds the .lessx file and returns the result as a string. Now usually the view engine will take controller name and method name and by convention look in the /Views/css/index.aspx file. It doesn’t quite make sense to put it in the views folder so I have overridden the default web forms view engine so that they can be placed in the /Public/Css folder. This can be configured to your liking (convention).
public class CustomViewEngine : WebFormViewEngine { public CustomViewEngine() { var locs = new List<string>(base.ViewLocationFormats); locs.Add("~/Public/{1}/{0}"); //My personal choice locs.Add("~/Views/{1}/{0}"); //An alternative choice base.ViewLocationFormats = locs.ToArray(); } } |
Once your Custom View Engine has been overridden, you will need to add it to the ViewEngines.Engines collection in the application start method in the global.asax.cs.
ViewEngines.Engines.Clear(); ViewEngines.Engines.Add(new CustomViewEngine()); |
This will then allow us to write a *.lessx file like the following; you can also pass a strongly typed model to the .lessx file if you so wish in the same way you would to a regular aspx view.
<%@ Page Language="C#" Inherits="System.Web.Mvc.ViewPage" %>
@basecolor: <%= ViewData["baseColor"] %>;
body {
background: @basecolor;
}
|
Which will be rendered to the browser as:
body {
background: #aabbcc;
}
|
And that’s it. I’m still working on the finer details but I am liking how it works at the moment. If you have any thoughts or suggestions, please leave a comment.
Adam
NoRM, MongoDb and Complex Queries
In my last post I told you about how you can use Regular Expressions to do complex queries in mongodb. In this post I will show you some more features of the Linq Provider.
Updated: 16th May 2010 after some feedback
Take these two classes for example:
public class User { public string Id { get; set; } public string Name { get; set; } public IList<Role> Roles { get; set; } } public class Role { public string Name { get; set; } public int AccessLevel { get; set; } } |
and given the following data:
var standardRole = new Role {Name = "Standard", AccessLevel = 2}; var adminRole = new Role { Name = "Standard", AccessLevel = 5 }; var user1 = new User() { Id = "jbloggs", Name = "Joe Bloggs", Roles = new List<Role> { standardRole, adminRole } }; var user2 = new User() { Id = "tsmith", Name = "Tony Smith", Roles = new List<Role> { standardRole, } }; |
Now say I want to find all users with access level greater than or equal to 5. In Linq-2-sql or plain sql your might write this query like the following.
var query = list.Where(x => x.Roles.Where(y => y.AccessLevel >= 5).Count() > 0).ToList(); |
This query will not work in the NoRM linq provider, however there is an alternative.
1. var queryforMongo = list.Where(x => x.Roles[0].AccessLevel >= 5).ToList(); 2. var queryforMongo = list.Where(x => x.Roles.Any(y=>y.AccessLevel >= 5)).ToList(); |
Query 1 here will only return Users where the first role has an access level greater than or equal to 5.
Query 2 will return Users where 1 of the Roles has an access level greater than or equal to 5.
One caveat though. If you try and do a complex query ( one involving an “or” condition or a “and” using the same property ) then you will get an exception. This is because complex queries get converted to a javascript function that cannot be used in the same ways.
Hope this comes in handy for folks out there.
Adam
NOSQL, NoRM (mongoDB) and Regular Expressions
Over the past few weeks I have got quite involved in the development of the Open Source NoRM project started by Andrew Theken which is a MongoDB driver for C#. In particular I have been refactoring and adding new functionality to the LINQ provider.
It currently supports a lot of functionality including deep queries, regex, datetime which is really exciting. In this post though I am going to concentrate on regular expressions.
This is the newest part of the Linq provider however it is probably the most powerful, especially for complex queries. The reason for this is that MongoDB will use the indexes created when a regular expression is used (where the query is not a complex query). A complex query is one that filters on the same property twice, uses a string function (replace/substring/toLower etc) or does some other fancy stuff. For example using the toUpper() method.
var products = session.Products.Where(x => x.Name.ToUpper() == "TEST3").ToList(); |
Anyways….i digress. So, here is an example of using a Regex in a Linq Query. Pretty simple.
var products = session.Products.Where(p => Regex.IsMatch(p.Name, "^te")).ToList(); |
Using the static Regex.IsMatch is the only way to invoke a regex call using the Linq Provider. This will however run blazingly fast. I tested this query on 1,000,000 Products and it only took 1.5sec, which was approximately 10x faster than when a complex query is invoked. There are however 3 string functions that have been optimized using this regex functionality. They are StartsWith(), EndsWith() and Contains() which is why the following query only takes 2secs to return over 48,000 rows, however when using Skip() and Take() you can get 50 results back in just milliseconds.
var products = session.Products.Where(x => x.Name.StartsWith("X")).ToList(); |
The Linq provider also supports 3 of the RegexOptions. They are RegexOptions.IgnoreCase (but please note this will not use the index so will be slower), RegexOptions.Multiline and RegexOptions.None. Regex’s can also be used in conjunction with other filters. eg.
var products = session.Products.Where(p => Regex.IsMatch(p.Name, "^te") && p.Price == 10).ToList(); |
This query is not considered a complex query because two different properties are used and an “and”(&&) operator is used.
Please note: Any time a “or” (||) operator is used, it will be considered a complex query.
Summary:
If you have a filter than can be written as a regex, chances are it will be as fast or faster than without using a regex.
So please go and try out NoRM and enjoy the freedom. I will try and post some more cool stuff in the LINQ provider over the next few weeks. Stay tuned.
FluentValidation Xval Integration
After a few months of using FluentValidation I asked its author Jeremy Skinner if it were possible to integrate this with xVal. At that time it was not possible because there were no easy way to access the properties needed by xVal. After submitting a few patches, we now have a solution which enables xVal integration with most of the FV validators.
It currently supports the following FV validatiors:
- NullValidator
- NotEmptyValidator
- LengthValidator
- RegularExpressionValidator
- ComparisonValidator including:
- Equal
- Not Equal
- Greater Than or Equal
- Less Than or Equal
To configure the integration we need to tell xVal to use the FV rules provider rather than the default one. This is done in the global.asax.cs in Application_Start().
|
The rules provider here is instructed to use the AttributedValidatorFactory to instruct the provider to use the attribute attached to the model class to find the validation class for that model.
Note: This is still new and xVal is still in beta so there may be some issues. If you find any please let me know so we can fix them as soon as possible.
Hopefully once its ready it can be checked in with the other providers at the xVal codeplex site.
This is currently in the development source code which can be downloaded and tried now.
Cheers,
Adam
Fluent Validation Model Binder – Asp.net MVC
A few weeks ago I found the Fluent Validation framework by Jeremy Skinner. I needed to conditionally validate a model depending on an application setting. eg. Description field mandatory / not mandatory depending on the clients business requirements. I loved the simplicity of the framework and the separation from the model it provided.
Since then I have submitted a few patches for the framework, one of which is the Fluent Validation Model binder. Inspired by the Data Annotations Model binder, it works in much the same way. Once you have it set to your default model binder, it will validate any model which contains the specific attribute. This will become clear in the examples below.
Firstly lets take our model.
[Validator(typeof(LineItemValidator))] public class LineItem { public int LineNumber { get; set; } public DateTime Date { get; set; } public string Description { get; set; } public decimal Net { get; set; } public decimal Tax { get; set; } public decimal Gross { get; set; } } |
Attached to this simple LineItem class is a Validator attribute. This attribute is used by the Model Binder to locate the Class used for validation. Below I will define my LineItemValidator class which will hold the rules for the validation.
|
As you can see the class must inherit from AbstractValidator<T> where T is the model you want to define the rules for. The first rule uses the conditional When clause where it will only validate that the field is NotEmpty when the Settings.DescriptionRequired boolean is true. Also, another thing is the complex validation taking place on the Gross field. Not only does it validate that it is greater than 0, but that the value is equal to the net + tax amount. This is very elegant indeed. I have also specified the ‘WithName’ clause which has also been integrated into the model binder so that if an error occurs with the gross field, the WithName value will be displayed when an error happens. This is extremely handy for language localization or when the name of the field on the Model is insufficient.
Wiring this Model binder up in the Application_start event in the global.asax.cs is as easy as this.
|
Note that we have to pass an instance of the AttributedValidatoryFactory into the Model Binder. This means if you have an alternate way of locating the Validator class other than via the attribute you can inherit from IValidatorFactory and create your own.
Now when a parameter of LineItem gets passed into a controller it will be validated against the model and all errors placed into the ModelState. This can then check the isValid property to determine if there are any errors and proceed accordingly.
[AcceptVerbs(HttpVerbs.Post)] public ActionResult Edit(List<LineItem> lineItems) { if (ModelState.IsValid) return RedirectToAction("Edit"); return View(); }
|
And that’s it. I hope you enjoy using the Fluent Validation framework as much as I have and happy coding. For all you guys waiting for xVal integration, I will try and post a solution in the coming week.
Adam