Oslo Installation - Windows Installer

I was installing the Oslo CTP on my XP laptop (I've had it on my Vista PC for over a month now), but during the installation I got an error telling me that I needed a newer Windows Installer, with no additional information.  I downloaded the Installer 4.5 Redistributable, and after a reboot now my Oslo SDK Install didn't do anything.  I went into Add/Remove programs, removed "Microsoft Codename Oslo Repository" and "Microsoft Codename Oslo SDK Temporary Setup Files" and kicked off the install again and this time it all went well.  Life in CTP, i guess...

Building dynamic where logic in LINQ

If you need to dynamically build and/or logic in your LINQ query, you can use predicates and the PredicateBuilder pattern described here.  My current client uses VB.NET so there really couldn't use the source that is provided in that article, so their only choice was going to be to reference the LINQKit assembly provided by the author (great C# book, by the way).  It seemed silly to me to have to include that assembly for such small functionality since they are not using any of its other features, it would be another assembly that would need to be added to their distributable.  So I translated the code to VB and added it to their project:

Imports System
Imports System.Linq
Imports System.Linq.Expressions
Imports System.Collections.Generic
Imports System.Runtime.CompilerServices
Public Module PredicateBuilder
    Public Function [True](Of T)() As Expression(Of Func(Of T, Boolean))
        Return Function(f) True
    End Function

    Public Function [False](Of T)() As Expression(Of Func(Of T, Boolean))
        Return Function(f) False
    End Function

    <Extension()> _
    Public Function [Or](Of T)(ByVal expr1 As Expression(Of Func(Of T, Boolean)), ByVal expr2 As Expression(Of Func(Of T, Boolean))) As Expression(Of Func(Of T, Boolean))
        Dim invokedExpr = Expression.Invoke(expr2, Enumerable.Cast(Of Expression)(expr1.Parameters))
        Return Expression.Lambda(Of Func(Of T, Boolean))(Expression.OrElse(expr1.Body, invokedExpr), expr1.Parameters)
    End Function

    <Extension()> _
    Public Function [And](Of T)(ByVal expr1 As Expression(Of Func(Of T, Boolean)), ByVal expr2 As Expression(Of Func(Of T, Boolean))) As Expression(Of Func(Of T, Boolean))
        Dim invokedExpr = Expression.Invoke(expr2, Enumerable.Cast(Of Expression)(expr1.Parameters))
        Return Expression.Lambda(Of Func(Of T, Boolean))(Expression.AndAlso(expr1.Body, invokedExpr), expr1.Parameters)
    End Function
End Module

 

It had been a long time since I did any VB coding so it was a pain to figure out at first, but got it to work, so the code above translates directly to the C# sample used in LINQKit.  I guess for the non-VB-speaking C# developers out there, as far as I can tell, a Module is a just a static class.

Once you have that in place you can do things like:

Dim predicate = PredicateBuilder.False(Of myType)()
predicate = predicate.Or(Function(p) p.StartDate = "1/1/2008")
If (includeEndDate) Then
    predicate = predicate.Or(Function(p) p.EndDate = "12/31/2008")
End If

Dim myList = From b In mycontext.myTable.Where(predicate).ToList()

Scrum session at CodeCamp

I spoke about Scrum at the Orlando .NET Code Camp today, it went really well.  You can download the presentation file here.

Getting build notifications from TFS

TFS ships with an event subscription tool that you can access through command line and it's located at "C:\Program Files\Microsoft Visual Studio 2008 Team Foundation Server\TF Setup".  You can use it to subscribe to a variety of TFS events dealing with builds, check-ins, projects, work items, etc.    You can subscribe to a handful of events from the team project node in team explorer, but this gives you a much bigger list.  In order to get notifications when the build fails or partially succeeds (which means that your assemblies built, but your tests failed), you would do the following:

bissubscribe /eventType BuildCompletionEvent /address you@email.com /deliveryType EmailHtml /server yourTfsServer /filter "TeamProject = 'TeamProjectName' AND (CompletionStatus='Failed' OR CompletionStatus='Partially Succeeded')"

There's also a UI for it in CodePlex that will allow you to do the same if you don't want to deal with command line.

MVC Image button

I am helping out with the Orlando .NET Code Camp website and they needed to add an image button, which is something that is not supported out of the box by the current release of MVC.  So after looking around I found BuildUrlFromExpression.  Unfortunately it had been moved in the latest release and I tracked it down to the Futures project in CodePlex.  Instead of being part of Html, it's under LinkBuilder now, this is how you use it:

<a href="<%= LinkBuilder.BuildUrlFromExpression<YourController>(null, Html.RouteCollection,  x => x.Action() ) %>">
<img src="images/img.gif" alt="Some image" style="border-style: none"/></a>

Which when clicked, would call into your controller and get routed correctly. 

After downloading the Futures assemblies, you need to reference the Microsoft.Web.Mvc assembly and add this to your config file, under <system.web> <pages> <namespaces>:

<add namespace="Microsoft.Web.Mvc" />

DataReader vs. Dataset vs. Linq

I'm currently working with a client that supports an application originally built around 2003 so it was targeting the 1.1 Framework.  Last summer they upgraded to the 3.5 Framework, but they have not yet taken advantage of the new features that can help them improve their performance.  Digging through their code, I found a few things that can be changed pretty easily, but before just telling them "do X because it will help you", I wanted to run some metrics based on their specific scenario.  So I picked one of their main tables and loaded it with different sets of data so that I could test the length of time that it takes to load records into memory for the following counts: 10, 100, 1k, 5k, 10k, 50k, 100k, 150k, 200k, 250k.  The table has 89 columns, with the majority of them being nullable (so need to account for DBNull when loading the data).  Currently they are using mostly Datasets and Strongly-typed datasets to load data, and they are using the OleDbProvider since they used to target more than just SQl Server.  The huge use of datasets can be attributed to Microsoft's big push for them back then...that was before generics so you either had to choose between ArrayList, your own collection type, or strongly-typed datasets to move data around your app.  So here are the results:

DataSet - OleDbDataAdapter: Use the data adapter to load records directly into a DataSet:

string _Sql = "SELECT * FROM TestTable";
DataSet _Dataset = new DataSet();

using (OleDbConnection _Connection = Connection.CreateOleDbConnection())
using (OleDbDataAdapter _Adapter = new OleDbDataAdapter(_Sql, _Connection))
{
    _Adapter.FillSchema(_Dataset, SchemaType.Source);
    _Adapter.Fill(_Dataset);
    _Adapter.SelectCommand.ResetCommandTimeout();
    _Adapter.SelectCommand.Cancel();
}

# Records - Execution Time (seconds) 10 - 0.008113 100 - 0.0455828 1,000 - 0.2481427 5,000 - 1.3727283 10,000 - 1.932733 50,000 - 6.6397684 100,000 - 12.5606824 150,000 - 18.1799086 200,000 - 23.8470934 250,000 - 29.2775204

DataSet - SqlDataAdapter: Same code as above but with a SqlConnection and a SqlDataAdapter. Just switching the adapter made a really big difference, one that you may not be able to feel if you are only dealing with one record as it's milliseconds difference, as the recordset grows, so does the difference.  And remember, I'm not running any fancy queries or doing anything special that the SQL Adapter would necessarily streamline, but it's obvious that it knows how to talk to SQL a lot better even on a simple scenario.

# Records - Execution Time (seconds)
10 - 0.0034504
100 - 0.0120853
1,000 - 0.1344648
5,000 - 0.6467623
10,000 - 0.9996195
50,000 - 3.7021627
100,000 - 7.5787567
150,000 - 10.7082539
200,000 - 13.7969211
250,000 - 17.1756977

Strongly typed-datasets: The results were about the same as using a "regular" dataset, with the added gain that now you can do compile-time checking of your code that references the data:

# Records - OleDb DataSet - SqlProvider
10 - 0.0036161 - 0.0260737
100 - 0.014285 - 0.1248127
1000 - 0.1254575 - 0.2927101
5000 - 0.7131587 - 1.3627991
10000 - 1.0339554 - 1.9029454
50000 - 3.8798861 - 6.8526659
100000 - 7.2328849 - 12.6700968
150000 - 10.278075 - 17.9946054
200000 - 13.6603168 - 23.75068
250000 - 17.0334383 - 29.7786064

SqlDataReader: I was definitely expecting this to be a lot faster, and it was.  But then I had to take into account that I want to provide the same functionality that a strongly-typed dataset gives you.  So I created a data-object that would get hydrated with the data that is returned with the DataReader and then load that into a List<T> which would then let me send the data back to the consumer so that it can be bound to a control or whatever else the consumer needs to do with it.  So even with all of that plumbing work (remember, I have to check for DBNull on most of the data returning), it was still faster than going with a DataSet.  I created an enum for my columns since accessing values by columnIndex is a lot faster than by name, so that's what TableOrdinals is.  GetString(), GetInt32(), and GetBoolean() are my helper methods that I used to check for DbNull and get the data out of the DataReader.

List<MyTestType> _TestList = new List<MyTestType>();
string _Sql = "SELECT * FROM TestTable";
using (SqlConnection _Connection = Connection.CreateSqlConnection())
using (SqlCommand _Command = new SqlCommand(_Sql, _Connection))
using (SqlDataReader _Reader = _Command.ExecuteReader(System.Data.CommandBehavior.CloseConnection))
{
    while (_Reader.Read())
    {
        _TestList.Add(
            new MyTestType
        {
            Name = GetString(_Reader, TableOrdinals.Name),
            AccountNumber = GetInt32(_Reader, TableOrdinals.AccountNumber), 
            IsEnabled = GetBoolean(_Reader, TableOrdinals.IsEnabled), 
            ...and so on...
        });
    }
}

# Records - OleDbDataReader(secs) - SqlDataReader(secs)
10 - 0.0031392 - 0.0014887
100 - 0.0226537 - 0.0095741
1000 - 0.2497585 - 0.1020875
5000 - 1.2553494 - 0.5808425
10000 - 1.7956469 - 0.7263065
50000 - 6.117392 - 2.3071203
100000 - 10.7574412 - 4.3612802
150000 - 15.3076586 - 6.1806721
200000 - 20.0983921 - 8.0577727
250000 - 25.3383307 - 10.1982922

Linq to SQL: I added LINQ to SQL classes through Visual Studio and I added my table from Server Explorer.  Then I used the DataContext to get to my data and returned all the data as a List.  Linq to SQL uses delayed execution so I had to do that in order for it to go through all the records and return them.  The results were faster than a DataReader, not by a lot, but with the added bonus that I don't have to create my data type and I don't have to worry about DbNull checking and other plumbing.

using (DataClasses1DataContext _Context = new DataClasses1DataContext())
{
    _Context.TestTableRecords.ToList();
}

# Records - Execution Time(secs)
10 - 0.188708
100 - 0.1554585
1000 - 0.1766154
5000 - 0.6223885
10000 - 0.7556139
50000 - 2.1826003
100000 - 3.8992804
150000 - 5.4793223
200000 - 7.5617141
250000 - 9.348437

Compiled Linq to SQL: So I thought I was done, afterall I had done what I had set out to do, but then one of my business partners Marius, pointed me to an article that mentioned Compiled Linq to SQL, which then led me to this MSDN blog post that describes what it is, so I gave it a shot and WOW!!!  Basically, it translates your code into SQL the first time and then keeps your generated SQL to be reused.  So using the same scenario:

private static Func<DataClasses1DataContext, IQueryable<TestTable>>
    SelectTestTableCompiledQuery = System.Data.Linq.CompiledQuery.Compile(

        (DataClasses1DataContext contextLocal) =>
            from q in contextLocal.TestTableRecords
            select q);

and then from within your data-gathering method:

using (DataClasses1DataContext _Context = new DataClasses1DataContext())
{
    SelectTop10CompiledQuery(_Context);                        
}

# Records - Execution Time(secs)
10 - 0.0066187
100 - 0.013831
1000 - 0.1380404
5000 - 0.4577587
10000 - 0.5304408
50000 - 1.1912905
100000 - 2.0506899
150000 - 2.7066448
200000 - 3.5848036
250000 - 4.2253995

So I can get a huge performance increase by taking advantage of the newer technologies, but just by making small changes such as using the right provider can make a difference.  Obviously, we can't just blindly start making changes left and right, use your best judgement and tackle areas where you will get the most benefit.  And always take a look at what's happening in SQL by using Profiler...Linq can sometimes generate some funny stuff!

Here are all the results and a nice picture:

What does it mean to be Agile?

I started working on a contract with a local software company where I am helping them adopt Scrum.  I've been "Scrumming" for over 2 years (yes, I am a ScrumMaster) and I really enjoy this type of work.  I'm not doing any coding (that's what my nights are for), I am pretty much helping them put the process in place and coaching them as they try to get a grip on the new methodology.  So as I'm going through the process, I am realizing more and more that just because you are following an agile process, it doesn't mean that you are Agile.

Being Agile means different things to different people, as a self-organizing team, the team is responsible for adopting agile software development techniques that will help them improve on what is delivered to the product owner as a sprint completes.  Examples of agile techniques include:
-    Test-driven development: Unit test cases are coded in order to execute new functionality.  Anyone can run the test cases at any time and verify that new functionality doesn’t break existing test cases.  Unit test cases can then be automated and executed as part of a build.  This practice does not do away or QA testing as it a developer’s responsibility to look after and run the test cases.  Test-driven development calls for a “test first” approach to development, but this can be adapted to include a unit test as part of the code deliverables.
-    Refactoring: Code duplication is avoided by consolidating and relocating duplicate functionality.
-    Continuous integration: As code is checked-in, a build automatically kicks off and the team is notified of any errors.  The build includes unit tests, and can run anywhere from nightly to as soon as the code is checked in.
-    Coding standards: Coding patterns and naming conventions are established and adhered to by the team, which makes cooperation between team members a lot easier.  A developer does not need to familiarize him/herself with the coding practices of a sub-system if the same standards are followed by everyone.
-    Automated acceptance/regression testing: By automating some of the regression testing, the team can spend more time testing new features and further automating testing.  A repeatable process is important to insure better quality in future releases.

The Agile Manifesto tells us that these are the values of Agile Development:
-    Value Individuals and Interactions  over Processes and Tools
-    Value Working Software over Comprehensive Documentation
-    Values Customer Collaboration over Contract Negotiation
-    Value Responding to Change over Following a Plan

 

So, there is a lot more to Agile than just adopting a process, but it's a great start!