There’s no doubt in my mind that code libraries and frameworks are fantastic for saving time and work, after all I want to spend time solving my business problem, not writing infrastructure.  Nowhere is this more true than it is with Object-Relational Mapping, or ORM, libraries. 

Broadly speaking, if you’re still writing application code that requires that you also write SQL, you’re wasting your time.  Wasting time thinking about SQL syntax.  Wasting time writing it.  Wasting time testing, debugging and maintaining it.

I believe this so much, but was so dissatisfied with any existing ORM offering, that I wrote my own ORM. It wasn’t a trivial task, but I have something that does exactly what I need, on the platforms I need, and does it at a speed that I consider more than acceptable.  Occasionally I hit a data storage requirement that ORMs, even my own, aren’t so good at. 

ORM usage is usually viewed as one of two approaches: code first or data first.  With the code-first approach, a developer defines the storage entity classes and the ORM generates a backing database from them.  With data first, the developer feeds a database into the ORM or an ORM tool, and it then generates the entity classes for you. 

But this doesn’t cover all scenarios – which is something no ORM that I’m aware of seems to acknowledge.  Consider the following use-case (and this is a real-world use case that I had to design for, not some mental exercise).

I have an application that allows users to define storage for data at run time in a completely ad-hoc manner.  They get to choose what data items they want to save, but even those data items are dynamically available so they are available only at run time.

So we need to store effectively a flat table of data with an unknown set of columns.  The column names and data types are unknown until after the application is running on the user’s machine.

So the entities are neither data first nor code first.  I’ve not thought of a catchy term for these types of scenarios, so for now I’ll just call it “user first” since the user has the idea of what they want to store and we have to accommodate that.  This is why I created support in the OpenNETCF ORM for the DynamicEntity.

Let’s assume that the user decides they wanted to store a FirstName and LastName for a person.  For convenience, we also want to to store a generated ID for the Person entities that get stored.

At run time, we generate some FieldAttributes that define the Person:

   1: var fieldList = new List<FieldAttribute>();

   2: fieldList.Add(new FieldAttribute()

   3: {

   4:     FieldName = "ID",

   5:     IsPrimaryKey = true,

   6:     DataType = System.Data.DbType.Int32

   7: });

   8:  

   9: fieldList.Add(new FieldAttribute()

  10: {

  11:     FieldName = "FirstName",

  12:     DataType = System.Data.DbType.String

  13: });

  14:  

  15: fieldList.Add(new FieldAttribute()

  16: {

  17:     FieldName = "LastName",

  18:     DataType = System.Data.DbType.String,

  19:     AllowsNulls = false

  20: });

And then we create and register a DynamicEntityDefinition with the DataStore:

   1: var definition = new DynamicEntityDefinition(

   2:                               "Person", 

   3:                               fieldList, 

   4:                               KeyScheme.Identity);

   5:  

   6: store.RegisterDynamicEntity(definition);

Now, any time we want to store an entity instance, we simply create a DynamicEntity and pass that to the Insert method, just like any other Entity instance, and the ORM handles storage for us.

   1: var entity = new DynamicEntity("Person");

   2: entity.Fields["FirstName"] = "John";

   3: entity.Fields["LastName"] = "Doe";

   4: store.Insert(entity);

   5:  

   6: entity = new DynamicEntity("Person");

   7: entity.Fields["FirstName"] = "Jim";

   8: entity.Fields["LastName"] = "Smith";

   9: store.Insert(entity);

The rest of the CRUD operations are similar, we simply have to name the definition type where appropriate.  FOr example, retrieving looks like this:

   1: var people = store.Select("Person")

Updating like this:

   1: var person = people.First();

   2: person.Fields["FirstName"] = "Joe";

   3: person.Fields["LastName"] = "Satriani";

   4: store.Update(person);

And Deleting like this

   1: store.Delete("Person", people.First().Fields["ID"]);

We’re no longer bound by the either-or box of traditional ORM thinking, and it leads to offering users some really interesting and powerful capabilities that before were relegated to only those who wanted to abandon an ORM and hand-roll the logic.

I’m (finally) trying to publish some better, more formal documentation for the ORM.  I just posted a simple example of how you can create a one-to-many relationship between Entities and how you can get the ORM to fill in the references when you do a Select call.  Take a look at the ORM documentation for more details.

I’ve just checked in new code changes and rolled a full release for the OpenNETCF ORM.  The latest code changes add transaction support.  This new release adds a load of features since the last (the last was way back in February), most notably full support for SQLite on all of the following platforms: Windows Desktop, Windows CE, Windows Phone and Mono for Android.

The OpenNETCF ORM has supported SQLite for a while now – ever since I needed an ORM for Android – but somehow I’d overlooked addeing SQLite support for the Compact Framework.  That oversight has been addressed and support is now in the latest change set (99274).  I’ve not yet rolled it into the Release as it doesn’t support Dynamic Entities yet, but that’s on the way.

The Data Collector feature in our Solution Family product line is one of the oldest (if not the oldest) sections of code, and as such it's in need of a refactor to improve how it works.  We've updated just about everything else that uses data storage to use the OpenNETCF ORM framework, but Data Collectors have languished, largely because they are complex.  The Data Collector lets a user create an ad-hoc data collection definition that translates into a SQL Compact Table at run time.  The problem with migrating to ORM is that the ORM requires compile-time definitions of all Entities.  At least that was the problem until today.

I just checked in a new change set that supports the concept of a DynamicEntity (along with all of the old goodness that is ORM).  Now you can create and register a DynamicEntityDefinition with your IDataStore at run time and it will generate a table for you in the back end.  New overloads for all of the typical CRUD commands (Select, Update, Insert, Delete) allow you to just ask for the DynamicEntity by name and it returns an array of DynamicEntity instances that hold the field names and data values.

It's in a "beta" state right now, but there's a test method in the change set that shows general usage of all CRUD operations.  Give it a spin, and if you find any bad behavior, report it on Codeplex.

We recently needed the ability to do most-recently-updated data caching in our Solution Family products.  Since the products use the OpenNETCF ORM Framework, it only made sense to update the framework itself to include events that fire whenever an Insert, Update or Delete occurs.  In fact I added Before and After versions for each. While I was at it, I also added a full complement of virtual On[Before|After][Insert|Update|Delete] methods to the DataStore base, allowing DataStore implementers to hook into the process as well.  I’m thinking I’ll use those at some point in the future to add some form of Trigger capabilities.

We've started doing work creating client applications for a few of our products to run on Android and iOS devices.  Since I'm a huge fan of code re-use, I took the time this week to port the OpenNETCF.ORM over to Xamarin's Mono for Android using a SQLite backing store implementation.  The biggest challenge was that SQLite doesn't support TableDirect not ResultSets, so it took a bit of code to get running.  Still, it took only a day and a half to get what I feel is pretty good support up and running.  I've not yet tested it through all of the possible permutations of queries, etc, but basic, single-table CRUD operations all test out fine.

So now a single code base can work on the Windows Desktop, Windows CE and Android (probably iOS and Windows Phone as well with very little work). If you're doing MonoDroid work, give it a try.

This post is part of my "Software Development" series. The TOC for the entire series can be found here.

Let’s start this post with a couple questions.  What is an ORM and why would you use one?  If you don’t know the answer to both of these questions, then this post is for you.  If you do know the answers, feel free to skip to the next post (after I’ve created it of course).

First, let’s look at what an ORM is. And ORM is an object-relational mapping.  According to Wikipedia, which we all know is the infallible and definitive resource for all knowledge, and ORM is “a programming technique for converting data between incompatible type systems in object-oriented programming languages.”  Ok, what the hell does that mean?  My definition is a little less “scholarly.”  I’d say that

An ORM is a way to abstract your data storage into simple class objects (POCOs if you’d like) so that you don’t have to worry about all the crap involved in actually writing data fields, rows, tables and the like.  As a developer, I want to deal with a “Person” class.  I don’t want to think about SQL statements.  I don’t want to have to worry about indexes or tables or even how my “Person” gets stored to or read from disk.  I just want to say “Save this Person instance” and have it done.  THAT is what an ORM is.  A framework that lets me concentrate on solving my business problem instead of spending days writing bullshit, mind-numbing, error-prone data access layer code.

Why would we use an ORM?  I think I’ve been fairly upfront that I consider myself to be a lazy developer.  No, I don’t mean that I take shortcuts or do shoddy work, I mean that I hate doing things more than once.  I hate having to write reams of code to solve problems that have already been solved.  I’ve been writing applications that consume data for years, and as a consequence, I’ve been writing data access code for years.  If you’re not using an ORM, you probably know down in your soul that this type of work flat-out sucks.  Anything that simplifies data access to me is a win.

Of course there are other, more tangible benefits as well.  If you’re using an ORM, it’s often possible to swap out data stores – so maybe you could write to SQL Server then swap a line of two in configuration and write to MySQL or an XML file.  It also allows you to mock things or create stubs to remove data access (really handy when someone tells you that your data access is what’s slowing things down when you’re pretty sure it’s not).

I can see that some of you still need convincing.  That’s good – you shouldn’t ever just take someone’s word for it that they know what they’re doing.  Ask for proof.  Well let’s look at a case of my own code, why I built the OpenNETCF ORM and how I was recently reminded why it’s a good thing.

A couple years ago I wrote an application for a time and attendance device (i.e. a time clock) that, not surprisingly, stored data about employees, punches, schedules, etc.  on the device.  It also has the option to store it on a server and synchronize the data from clock to server, but that’s not core to our discussion today.  The point is that we were storing a fair bit of data and this was a project I did right before I create the ORM framework.  It was, in fact, the project that made it clear to me that I needed to write an ORM.

Just about a month ago the customer wanted to extend the application, adding a couple features to the time clock that required updates to how the data was stored.  It took me very little time to realize that the existing DAL code was crap.  Crap that I architected and wrote.  Sure, it works.  They’ve shipped thousands of these devices and I’ve heard no complaints and had no bug reports, so functionally it’s fine and it does exactly what was required.  Nonetheless the code is crap and here’s why.

First, let’s look at a table class in the DAL, like an Employee (the fat that the DAL knows about tables is the first indication there’s a problem):

internal class EmployeeTable : Table 
{ 
    public override string Name { get { return "Employees"; } }

    public override ColumnInfo KeyField 
    { 
        get  
        { 
            return new ColumnInfo { Name = "EmployeeID", DataType = SqlDbType.Int }; 
        } 
    }

    internal protected override ColumnInfo[] GetColumnInfo() 
    { 
        return new ColumnInfo[] 
        { 
            KeyField, 
            new ColumnInfo { Name = "BadgeNumber", DataType = SqlDbType.NVarChar, Size = 50 }, 
            new ColumnInfo { Name = "BasePay", DataType = SqlDbType.Money }, 
            new ColumnInfo { Name = "DateOfHire", DataType = SqlDbType.DateTime }, 
            new ColumnInfo { Name = "FirstName", DataType = SqlDbType.NVarChar, Size = 50 }, 
            new ColumnInfo { Name = "LastName", DataType = SqlDbType.NVarChar, Size = 50 }, 
            // ... lots more column definitions ... 
        }; 
    }

    public override Index[] GetIndexDefinitions() 
    { 
        return new Index[] 
        { 
            new Index 
            { 
                Name = "IDX_EMPLOYEES_EMPLOYEEID_GUID",  
                SQL = "CREATE INDEX IDX_EMPLOYEES_EMPLOYEEID_GUID ON Employees (EmployeeID, GUID DESC)" 
            }, 
        }; 
    } 
}

So every table for an entity derives from Table, which looks like this (shorted a load for brevity in this post):

public abstract class Table : ITable 
{ 
    internal protected abstract ColumnInfo[] GetColumnInfo();

    public abstract string Name { get; } 
    public abstract ColumnInfo KeyField { get; }

    public virtual string GetCreateTableSql() 
    { 
        // default implementation - override for different versions 
        StringBuilder sb = new StringBuilder(); 
        sb.Append(string.Format("CREATE TABLE {0} (", Name));

        ColumnInfo[] infoset = GetColumnInfo(); 
        int i; 
        for (i = 0; i < (infoset.Length - 1); i++) 
        { 
            sb.Append(string.Format("{0},", infoset[i].ToColumnDeclaration())); 
        }

        sb.Append(string.Format("{0}", infoset[i].ToColumnDeclaration())); 
        sb.Append(")");

        return sb.ToString(); 
    }

    public virtual Index[] GetIndexDefinitions() 
    { 
        return null; 
    }

    public virtual IDbCommand GetInsertCommand() 
    { 
        SqlCeCommand cmd = new SqlCeCommand(); 

        // long, manual generation of the SQL and then creating the command

        return cmd; 
    }

    public IDbCommand GetUpdateCommand() 
    { 
        SqlCeCommand cmd = new SqlCeCommand(); 

        // long, manual generation of the SQL and then creating the command 

        return cmd; 
    }

    public IDbCommand GetDeleteCommand() 
    { 
        SqlCeCommand cmd = new SqlCeCommand(); 

        // long, manual generation of the SQL and then creating the command 

        return cmd; 
    }

Sure, I get a few bonus points for using inheritance so that each table doesn’t have to do all of this work, but it’s still a pain.  Adding a new Table required that I understand all of this goo, create the ColumnInfo right, know what the index stuff is, etc.  And what happens when I need to add a field to an existing table?  It’s not so clear.

Now how about consuming this from the app?  When the app needs to get an Employee  you have code like this:

public IEmployee[] GetAllEmployees(IDbConnection connection) 
{ 
    List list = new List();

    using (SqlCeCommand command = new SqlCeCommand()) 
    { 
        command.CommandText = EMPLOYEES_SELECT_SQL2; 
        command.Connection = connection as SqlCeConnection;

        using (var rs = command.ExecuteResultSet(ResultSetOptions.Scrollable | ResultSetOptions.Insensitive)) 
        { 
            GetEmployeeFieldOrdinals(rs, false);

            while (rs.Read()) 
            { 
                IEmployee employee = EntityService.CreateEmployee();

                employee.BadgeNumber = rs.IsDBNull(m_employeeFieldOrdinals["BadgeNumber"]) 
                    ? null : rs.GetString(m_employeeFieldOrdinals["BadgeNumber"]); 
                employee.BasePay = rs.IsDBNull(m_employeeFieldOrdinals["EmployeeNumber"]) 
                    ? 0 : rs.GetDecimal(m_employeeFieldOrdinals["BasePay"]); 
                employee.DateOfHire = rs.IsDBNull(m_employeeFieldOrdinals["DateOfHire"]) 
                    ? DateTime.MinValue : rs.GetDateTime(m_employeeFieldOrdinals["DateOfHire"]); 
                employee.EmployeeID = rs.GetInt32(m_employeeFieldOrdinals["EmployeeID"]); 
                employee.FirstName = rs.GetString(m_employeeFieldOrdinals["FirstName"]); 
                employee.LastName = rs.GetString(m_employeeFieldOrdinals["LastName"]); 
                string middleInitialStr = !rs.IsDBNull(m_employeeFieldOrdinals["MiddleInitial"]) 
                    ? rs.GetString(m_employeeFieldOrdinals["MiddleInitial"]) : String.Empty; 
                employee.MiddleInitial = String.IsNullOrEmpty(middleInitialStr) ? '\0' : middleInitialStr[0]; 

                 // on and on for another 100 lines of code) 

                 list.Add(employee); 
            } 
        } 
    }

    return list.ToArray(); 
} 

Nevermind the fact that this could be improved a little with GetFields – the big issues here are that you have to hard-code the SQL to get the data, then you have to parse the results and fill out the Employee entity instance.  You do this for every table.  You change a table, you then have to go change the SQL and every method that touches the table.  The process is error prone, time consuming and just not fun.  It also makes me uneasy because the test surface area needs to be big.  How do I ensure that all places that access the table were fixed?  Unit tests help give me some comfort, but really it has to go through full integration testing of all features (since Employees are used by just about every feature on the clock).

Now what would the ORM do for me here?  Without going into too much detail on exactly how to use the ORM (we’ll look at that in another blog entry), let’s just look at what the ORM version of things would look like.

We’d not have any “Table” crap.  No SQL.  No building Commands and no parsing Resultsets.  We’d just define an Entity like this:

[Entity] 
internal class Employee 
{ 
    [Field(IsPrimaryKey = true)] 
    public int EmployeeID { get; set; }        
    [Field] 
    public DateTime DateOfHire { get; set; } 
    [Field] 
    public string FirstName { get; set; } 
    [Field] 
    public string LastName { get; set; }

    // etc. 
}

Note how much cleaner this is than the Table code I had previously.  Also note that this one class replaces *both* the Table class and the Business Object class.  So this is much shorter.

What about all of that create table, insert, update and delete SQL and index garbage I had to know about, write and maintain?  Well, it’s replaced with this:

m_store = new DataStore(databasePath);

if (!m_store.StoreExists) 
{ 
    m_store.CreateStore(); 
}

m_store.AddType<Employee>(); 

That’s it.  Adding another Entity simply requires adding just one more line of code – a call to AddType for the new Entity type.  In fact the ORM can auto-detect all Entity types in an assembly with a single call if you want.  So that’s another big win.  The base class garbage gets shifted into a framework that’s already tested.  Less code for me to write means more time to solve my real problems and less chance for me to add bugs.

What about the long, ugly, unmaintainable query though?  Well that’s where the ORM really, really pays off.  Getting all Employees becomes stupid simple.

var allEmployees = m_store.Select<Employee>();  

Yep, that’s it. There are overloads that let you do filtering.  There are other methods that allow you to do paging.  Creates, updates and deletes are similarly easy. 

Why did I create my own instead of using one that already exists?  Simple – there isn’t one for the Compact Framework.  I also find that, like many existing IoC frameworks, they try to be everything to everyone and end up overly complex.  Another benefit to the OpenNETCF ORM is that it is fully supported on both the Compact and Full Frameworks, so I can use it in desktop and device projects and not have to cloud my brain with knowing multiple frameworks.  I even have a partial port to Windows Phone (it just needs a little time to work around my use of TableDirect in the SQL Compact implementation). 

Oh, and it’s fast.  Really fast.  Since my initial target was a device with limited resources, I wrote the code for that environment.  The SQL Compact implementation avoids using the query parser whenever possible because experience has taught me that as soon as you write actual SQL, you’re going to pay an order of magnitude performance penalty (yes, it really is that bad).  It uses TableDirect whenever possible.  It caches type info so reflection use is kept to a bare minimum.  It caches common commands so if SQL was necessary, it at least can reuse query plans.

So that’s why I use an ORM. Doing data access in any other way has become insanity.

This post is part of my "Software Development" series.  The TOC for the entire series can be found here.

Developing good software is hard.  Really hard.  Sure, anyone can buy a book on writing software or pull up some code samples and get something that compiles and runs, but that’s not’s really developing software.  A lot of code in the wild – I’d bet a vast majority of it – just plain sucks.

It’s hard to point out where the blame lies.  It seems that most developers are environmentally or institutionally destined to write bad code. Schools teach how to write code, but not how to architect it or to follow reasonable design practices.  In the zeal for clarity, publishers churn out books, blogs and samples that show bad practices (when is it ever a good idea to access a data model from your UI event handler?).  Managers and customers alike push hard to get things done now, not necessarily done right – only to find months or years later that doing it right would have saved a boatload of time and money.  And let’s face it – many developers are simply showing up to work to pull a paycheck.  You know who they are.  You’ve no doubt worked with them in the past.  You’re probably working with them now.

I was watching Gordon Ramsay the other day and it occurred to me that he and I are alike in our own peculiar way.  I’m not saying that I see myself as the “Gordon Ramsay of Software Development” – hardly -   but we share a common trait.  Just as Gordon gets angry and starts spewing colorful language when he walks into a crap kitchen, it bothers the hell out of me to see complete idiots in my chosen field out there just making a mess of things.  When I see bad code – not necessarily minor errors, or code that could be refactored and made better – but just outright shit code that should not have occurred to a developer in the first place it pisses me off.  By the nature of my work, often getting called in only when the project is off the rails, I see it all the time. Code that, on review, a peer or mentor should have seen and said “Whoa!  There’s no way that’s going into our code base”.  Code that just makes it harder for the next person to do their job.

In an effort to simplify things for my own code, for my customers’ code as well as anyone who is willing to listen to my ravings, I’ve spent a lot of time building, testing, fixing and extending tools and frameworks that many of which I turn around and give away.  This isn’t out of altruism, no, it’s largely because I’m a lazy developer.  I hate writing the same thing twice.  When I start a project, I don’t want to spend large amounts of time building up the same infrastructure that every project needs. Building up a framework for handling UI navigation isn’t what I’d call interesting, but just about every project needs it.  Handling object dependencies and events is common.  Writing a DAL for serializing and deserializing entities is not just drudgery, I find it’s highly susceptible to errors because you end up doing a lot of copy and paste.

I have these cool tools and frameworks that I use in literally every project I work on now.  That’s great for me, but it doesn’t really help others, right?  Without reasonable documentation or explanation, only a small handful of people are going to go through the effort of getting the tools and trying to understand them – even if they are deceptively simple and could potentially save you weeks of effort. 

So I’ve decided to put together a series of blogs over the coming weeks and months that explain, hopefully in simple terms, what these frameworks do, how to use them, and most importantly, why they are worth using.  There’s nothing groundbreaking here.  I didn’t invent some new way to do things.  I’ve simply appropriated other peoples’ ideas and extended them to work in the environments that I work.

Generally I’ll be covering the following topics and frameworks:

• Dependency Injection and Inversion of Control (using OpenNETCF IoC)
• Event Aggregation (using OpenNETCF IoC)
• Plug-in Architectures and interface-based programming (using OpenNETCF IoC)
• Software features as services (using OpenNETCF IoC)
• Data Access through an ORM (using OpenNETCF ORM)
• Parameter Checking (using OpenNETCF Extensions)
• Exposing data services over HTTP (using Padarn)
• Whatever else I think of

If there’s a topic you’d like me to talk about, feel free to send me an email.  I may turn on comments here and let you post ideas, but I find that when I enable comments on my blog, I start getting more comment spam than I really want to deal with, so if comments are turned off just drop me a line.

I checked in a load of updates to the ORM project earlier this week.  I've now published these changes, plus support for byte[] Fields and some other infrastructure work as an official release (build 1.0.11006).

Occasionally we need to store data in a database that doesn't neatly fit into the "normal" column type definitions of a database - for example we might want to store an image or just a serialized object. The newest OpenNETCF.ORM Library provides a simple mechanism for handling these types of objects, it's just not all that clear from the object model how it works. I'm hoping that this post will clarify things.

First, ORM can handle *any* type of object that is marked as a Field in an Entity class. If the Field is an Object, however, you have to help it out by providing a mechanism to serialize and deserialize that object.

Let's look at a simple example. Let's say I have the following Entity definition:

[Entity(KeyScheme.Identity)]
public class TestTable
{
    [Field(IsPrimaryKey = true)]
    public int TestID { get; set; }

    [Field]
    public CustomObject CustomObject { get; set; }
}

Sure, it's likely going to have more fields, but what's important here is the CustomObject Property that is a CustomObject class. What happens when the ORM library hits one of these when it's doing an Insert or Select?

If the object boils down to being an object, it calls either a Serialize or Deserialize method, depending on which it needs, in the Entity instance itself. The expectation is that these methods will have the following signatures (yes, I realize these should maybe be defined in some abstract base class, but for now they aren't):

[Entity(KeyScheme.Identity)]
public class TestTable
{
    [Field(IsPrimaryKey = true)]
    public int TestID { get; set; }

    [Field]
    public CustomObject CustomObject { get; set; }

    public byte[] Serialize(string fieldName)
    {
        // serialize
    }

		public object Deserialize(string fieldName, byte[] data)
    {
        // deserialize
    }
}

What happens here is that a string representation of the Field being requested will be passed in by the framework, and it's up to your implementation to correctly handle it. Let's extend this example by defining our CustomObject like this:

public class CustomObject
{
    public string ObjectName { get; set; }
    public Guid Identifier { get; set; }
    public int SomeIntProp { get; set; }

    public CustomObject()
    {
    }

    public CustomObject(byte[] data)
    {
        // deserialization ctor
        int offset = 0;

        // get the name length
        var nameLength = BitConverter.ToInt32(data, offset);

        // get the name bytes
        offset += 4; // past the length
        this.ObjectName = Encoding.ASCII.GetString(data, offset, nameLength);

        // get the GUID
        offset += nameLength;
        byte[] guidData = new byte[16];
        // we must copy the data since Guid doesn't have a ctor that allows us to specify an offset
        Buffer.BlockCopy(data, offset, guidData, 0, guidData.Length);
        this.Identifier = new Guid(guidData);

        // get the int property
        offset += guidData.Length;
        this.SomeIntProp = BitConverter.ToInt32(data, offset);
    }

    public byte[] AsByteArray()
    {
        List buffer = new List();

        byte[] nameData = Encoding.ASCII.GetBytes(this.ObjectName);

        // store the name length
        buffer.AddRange(BitConverter.GetBytes(nameData.Length));

        // store the name data
        buffer.AddRange(nameData);

        // store the GUID
        buffer.AddRange(this.Identifier.ToByteArray());

        // store the IntProp
        buffer.AddRange(BitConverter.GetBytes(this.SomeIntProp));

        return buffer.ToArray();
    }
}

Notice that I've decided to create my own custom serialization routines for the object. You could just as easily use a built-in serializer or whatever you'd like. I've put the serialization routines in the custom object class itself to prevent cluttering up my Entity definition. Now the Entity definition get's fleshed out to look like this for the Serialize and Deserialize methods:

[Entity(KeyScheme.Identity)]
public class TestTable
{
    [Field(IsPrimaryKey = true)]
    public int TestID { get; set; }

    [Field]
    public CustomObject CustomObject { get; set; }

    public byte[] Serialize(string fieldName)
    {
        if (fieldName == "CustomObject")
        {
            // This will always be true in this case since CustomObject is our only
            // Object Field.  The "if" block could be omitted, but for sample 
            // clarity I'm keeping it
            if (this.CustomObject == null) return null;
            return this.CustomObject.AsByteArray();
        }

        throw new NotSupportedException();
    }

		public object Deserialize(string fieldName, byte[] data)
    {
        if (fieldName == "CustomObject")
        {
            // This will always be true in this case since CustomObject is our only
            // Object Field.  The "if" block could be omitted, but for sample 
            // clarity I'm keeping it
            return new CustomObject(data);
        }

        throw new NotSupportedException();
    }
}

Pretty straightforward, especially if you only have one Object Field to worry about. Once the Serialize/Deserialize methods are implemented in the Entity, the new Entity works just like any other entity in ORM, so you can do something like this to insert a row and retrieve it back:

    var newObject = new CustomObject
    {
        ObjectName = "Object A",
        Identifier = Guid.NewGuid(),
        SomeIntProp = 12345
    };

    var testRow = new TestTable
    {
        CustomObject = newObject
    };

    Store.Insert(testRow);

    var existing = Store.Select().First();

I've found this to be extremely useful in a project where I had to store spectrum data for a row. Keep in mind that if the object is of any size, it could seriously impact your Select performance since the ORM has to select and rehydrate all of these objects for your returned data set. Use it sparingly, and if the object is often empty, it might pay off to put the custom object into its own child table and pull it in as a relationship so that the Serialize/Deserialize routines only get called when absolutely necessary.

We've shipped a first version of a customer project.  That's always good news, but the benefit to the community at large is that updates, improvements and fixes to both the ORM and IoC projects from that project have now propagated to the public code bases.

The IoC changes are pretty minor, which tells me that it's a pretty robust and mature library on the whole.

ORM had a load of changes in the SQL Compact implementation.  The interface for the DataStore has expanded by several methods due to use-cases I needed methods for and it's got a whole lot of performance improvements added.  ORM and is now shipping to real-world customers in a handheld product, so I consider it "release" quality.

Be aware that neither project has these changes rolled into a release package yet, so if you want these changes, grab the latest change set from the "Source Code" tab on the appropriate project page.

I've published a new release of the OpenNETCF.ORM library.  Some notable additions are:

• The ORM can now detect added Entity fields and automatically add the underlying columns to the store
• A new Select override can now filter result sets by multiple fields
• I've added a skeleton for a SQLite/Windows Phone 7 project with some implementation

If you have a desire to help me get the WinPhone implementation completed, I really could use the extra help.  I'm pretty busy, and without external help on this I don't see it getting implemented any time soon (unless we get hired to do a WinPhone project of course).

If you're not up on what the OpenNETCF ORM library is, in short it's an open-source ORM that actually works on the CF (NHibernate and Entity Framework do not). 

I've made a small update to the ORM library.  I've added a SQLite assembly and started implementeing the SQLiteDataStore class.  Note that I say "started" meaning that it's not done and ready for use (far from it in fact).  I did do a rough pass for store (database and table) creation.  The idea is that it gives at least some seed code for anyone interested in getting ORM up and running on their Windows Phone using SQLite.  If you are interested in doing this, please let me know.  I don't really have the time to devote to getting this done, so I'm looking for a capable developer who would like to take this on.

I've re-released the OpenNETCF.ORM library.  In my haste to port the code out of the production environment I'm using it in back to the Codeplex tree, I missed a file removal.  This broke the Codeplex release and, of course, I didn't check it before publishing the release.  I've rectified the problem and everything in Codeplex should now build.  If it doesn't send me an email or add a bug to the project to let me know.

I've been using the OpenNETCF.ORM library on a shipping project for a while now.  As expected, as I add features to the product, I've found problems and limitations with the ORM that I've addressed.  This morning I merged that branch back with the trunk available on Codeplex, so there's a whole new set of code available.  New features include:

• Better handling of reference fields
• Cascading inserts
• Cascading deletes
• Expanded capabilities for filtering on deletes
• Added support for more data types, including the "object" type
• Support for ROWGUID column

What it really needs now is a definitive sample application and documentation.  If you'd like to volunteer to work on either, I'd really appreciate it. 

Codeplex is in the process of upgrading their servers to TFS2010.  It isn't very clear, however, how to attach to the upgraded servers from older versions of Studio (like Studio 2008, which is required for all device development).  The answer is that you have to install a "forward compatibility update".

I've written about your DWR before.   While deleting code is certainly a way to increase it, it's not really the entire point.  The real point is that you should be writing less code.  I'm a huge fan of writing less code - especially less of the redundant, mind-numbing, why-can't-I-hire-an-intern-to-write-this code.  You know what I'm talking about - all of that data access layer garbage that we churn out to get our class data into and out of our relational databases.

If your job is primarily working on one application, you probably don't do this a whole lot, but when you are frequently starting new projects, you find yourself getting into this tedious stuff frequently, and I really, really hate doing it.  Not only is it torture, it's highly prone to errors since it's typically a copy, paste and adjust-the-names process.  How often have you have to write code to generate your database?  Does code like this look familiar?

public Book[] GetAllBooks()
{
  var books = new List<Book>();

  using (SqlCeCommand cmd = new SqlCeCommand("SELECT * FROM Book", Connection))
  {
    using (var results = cmd.ExecuteResultSet(ResultSetOptions.Insensitive))
    {
      while (results.Read())
      {
        if (m_bookOrdinals.Count == 0)
        {
          for (int i = 0; i < results.FieldCount; i++)
          {
            m_bookOrdinals.Add(results.GetName(i), i);
          }
        }

        books.Add(new Book
        {
          BookID = results.GetInt32(m_bookOrdinals["BookID"]),
          AuthorID = results.GetInt32(m_bookOrdinals["AuthorID"]),
          Title = results.GetString(m_bookOrdinals["Title"])
        });
      }
    }
  }

  return books.ToArray();
}

Well, I finally got tired of it and decided to spend some time writing code that would free me from having to do that stuff any longer.  The result is a new, open-source project called the OpenNETCF.ORM Framework.  It's a simple, lightweight ORM that helps take care of this tedium.  For example, the above block of code now looks like this:

public Book[] GetAllBooks()
{
  return Store.Select<Book>();
}

And it pulls all of the books from the underlying SQLCE database. Yes, it's that simple.  Of course getting it to do that requires a little infrastructure work.  For this one the Book class has to look like this:

[Entity]
public class Book
{
  [Field(IsIdentity=true, IsPrimaryKey=true)]
  public int BookID { get; set; }

  [Field]
  public int AuthorID { get; set; }

  [Field]
  public string Title { get; set; }

  [Field(SearchOrder=FieldSearchOrder.Ascending)]
  public BookType BookType { get; set; }
}

But that's a small price to pay in my book.  We went from 11 lines of code (not counting brackets) to one.  Multiply that out by the number of entities, filtering logic, paging logic and all the other code you typoically write and you'd greatly decreased your LOC count.

Now I'm not saying that OpenNETCF.ORM does everything that something like the Entity Framework does.  Remember, this is a small scale framework (the core is only 14k and the SqlCE implementation adds 17k) designed for mobile and embedded systems.  It's also put together by a team of one and as a side project while doing other work.

What it does have, though, is performance. It's actually faster than using direct SQL calls in many cases because it avoid the query processor whenever it can).  It has extensibility. The source code has a full implementation for SQL CE but I've also included the skeleton for an XML implementation for anyone who wants to try their hand at it, and it would be pretty easy to do a MySQL or SQLite implementation as well.  Most important, though, is that it has my commitment.  Like the IoC framework, I've already rolled ORM into a production application.  That means that as I find problems, they are going to get fixed.  As I find features that are missing that would help me get my job done, I'm going to add them.  Bottom line is that this is not a science project that I'm doing purely for fun and that will get abandoned when I get bored with it.

So if you're as tired of writing DAL code as I am, give it a try.  If you like it, let me know, or better yet update the docs or a new implementation.

If you're ready to get started, pull down the latest code (there are no releases quite yet) and look at the test project.  There should be enough to get you going on how to use it, but if you have questions, feel free to post them.