# Friday, May 28, 2010
Improved Rendering in Padarn

Every time we start a new project using our Padarn Web Server, I always like to see if we can improve the server's capabilities to make our job a little easier.  On the last project that turned into adding custom IHttpHandler support so we could easily serve up REST-based web services.

Well we started a new project last week, and unlike the last one, this one is UI-focused (the last one used Padarn strictly for services).  With that in mind, I went about looking at how to improve and simply building pages.

Of course I came back to our lack of support for server controls.  I did a little more playing and decided once again that it's a gigantic amount of work and it requires a boatload of RegEx and string manipulation - not necessarily a recipe for success on a low-memory, slow-CPU Windows CE device.

Still, there had to be other thing that full-blown ASP.NET offers that would be reasonable to add and that would help.  The existing samples we'd created in the past used overrode Page_Load method and then went about doing Response.Writes to emit page code.  The first time I did it I found it painful, so I created a class library that had objects for a lot of the basic HTML constructs and used that to at least help in generating well-formed code.

Still, I didn't really like the object model - it was still way too verbose - and it wasn't really like what anyone does on the desktop.  Additionally, rending in Page_Load really didn't feel right.

So I started looking around and found that the ASP.NET Page class provides a Render method.  That sounds a lot more like where I should be, oh maybe *rendering* a page.  Of course it required implementing the HtmlTextWriter, but that wasn't too terrible.

Once I implemented the Render method and started using the HtmlTextWriter, I quickly concluded that it, too, really sucked for generating page code.  It's just crazy verbose, it doesn't read anything like the page you're attempting to render and it's got things that just feel plain wrong (like having to add attributes before the tag they apply to? WTF is that?).

So I started looking at how ASP.NET developers who render their own controls with this monstrosity got around in.  It turns out that there is a pretty nice fluent interface to the HtmlTextWriter available on CodePlex.  It was missing a lot of stuff that I needed (and had a load of non-HtmlTextWriter stuff I didn't need), but it was a great start.  So after expanding the object model there I can now render a page in something that doesn't give me quite so much heartburn.

What originally looked like this:

protected override void Page_Load(object sender, EventArgs e)
{
  Response.Write(DoctypeTags.XHTML1);

  Response.Write("<html xmlns=\"http://www.w3.org/1999/xhtml\">");
  Response.Write("<head>");
  Response.Write("<link rel=\"stylesheet\" type=\"text/css\" href=\"css/mystyle.css\" />");
  Response.Write("</head>");

  Response.Write("<body>");

  Response.Write("<h1>");
  Response.Write("Hello World!");
  Response.Write("</h1>");

  Response.Write("</body>");
  Response.Write("</html>");
  Response.Flush();
}

Ends up looking more like this:

protected override void Render2(HtmlTextWriter writer)
{
  writer.Write(DoctypeTags.XHTML1);

  writer
    .Tag(HtmlTextWriterTag.Html, t => t["xmlns", "http://www.w3.org/1999/xhtml"])

      .Tag(HtmlTextWriterTag.Head)
        .StylesheetLink("/css/mystyle.css")
      .EndTag() // head

      .Tag(HtmlTextWriterTag.Body)
        .H1("Hello World!")
      .EndTag() // body
    .EndTag(); // html
}

And it gets much cleaner looking the larger the page is.  I also find I'm doing a lot less searching for where I malformed my HTML.

I think the next Item I'll look into is the __doPostBack Javascript method that IIS injects into pretty much every page.  It's probably the next incremental step toward better productivity.

OpenNETCF | Padarn | Patterns and Practices
Friday, May 28, 2010 6:43:06 PM (Central Daylight Time, UTC-05:00)  #     | 
# Thursday, May 27, 2010
Why you should use EventHandler.GetInvocationList

Introduction

Some time ago I explained to one of my friends the benefits of using GetInvocationList when raising events over simply calling the event handler delegate directly.  He understood my explanation and they implemented their events using the pattern.

Well today he was asked by another engineer why the code is always using "this GetInvocationList call for events" and so I figured I could revisit the explanation a little more formally both for his benefit as well as the other 3 people that read this blog.

The Challenge of Multicast Delegates

An event is, for all intents and purposes, a muticast delegate.  What that means is that you can have 0:n listeners listening for that event.  When you get you call the delegate directly in your code, the CLR iterates through all of the listeners and calls them for you without you having to know how many listeners there are.  This is all well and good *when things go right* but the problem is that things don't always go right.  This is especially the case if you're writing a class that will be consumed by someone else and you have no control over what their handler might be doing.

What happens if a handler throws an exception?  Well the CLR stops iterating through delegates an raises that exception to the method that invoked the delegate.  All handlers that were in the delegate list after the exception *will never see the event*.

To be a little more concrete, let's look at an example.

Let's say we have a class that publishes an event.  We'll call it Publisher and name the event the very original name "MyEvent".

class Publisher
{
  public event EventHandler MyEvent;

  public Publisher()
  {
  }
}

Now when we want to raise that event, the most common thing I've seen is to just get a handler and call it (ensuring it's not null).  Something like this:

public void RaiseDirect()
{
  var handler = MyEvent;
  if (handler == null)
  {
    Console.WriteLine("No listeners");
    return;
  }

  try
  {
    handler(this, null);
  }
  catch(Exception)
  {
    Console.WriteLine("A listener threw an exception in its handler");
  }
}

Now let's look at where this falls down.  First let's create a couple listeners.  Good ones that successfully run an EventHandler and bad ones that throw an exception in their EventHandler:

class Listener
{
  static int m_number = 0;

  public Listener()
  {
    m_number++;
    Name = this.GetType().Name + m_number.ToString();
  }

  public string Name { get; private set; }
}

class GoodListener : Listener
{
  public GoodListener(Publisher publisher)
  {
    publisher.MyEvent += new EventHandler(publisher_MyEvent);
  }

  void publisher_MyEvent(object sender, EventArgs e)
  {
    Console.WriteLine(string.Format("{0} handled event", Name));
  }
}

class BadListener : Listener
{
  public BadListener(Publisher publisher)
  {
    publisher.MyEvent += new EventHandler(publisher_MyEvent);
  }

  void publisher_MyEvent(object sender, EventArgs e)
  {
    Console.WriteLine(string.Format("{0} threw in handler", Name));
    throw new Exception("failure");
  }
}

Now let's wire these up and see what happens with some quick test code:

List<Listener> m_listeners = new List<Listener>();

Publisher publisher = new Publisher();

for (int i = 0; i < 5; i++)
{
  if (i % 3 == 0)
  {
    m_listeners.Add(new BadListener(publisher));
  }
  else
  {
    m_listeners.Add(new GoodListener(publisher));
  }
}

Console.WriteLine("Direct event");
Console.WriteLine("------------");
publisher.RaiseDirect();

We run this and we get the following:

directevent.png

You'll see that only 1 handler (a bad one) ran.  It threw an exception and no one else knew jack about the event.

Using the Invocation List

Now let's look at how we get past this.  The EventHandler delegate exposes a method called GetInvocationList which returns an array of all of the delegates.  Using this we can iterate across all of the EventHandlers manually.  Something like this:

public void RaiseIterative()
{
  var handler = MyEvent;
  if (handler == null)
  {
    Console.WriteLine("No listeners");
    return;
  }

  foreach (EventHandler h in handler.GetInvocationList())
  {
    try
    {
      h(this, null);
    }
    catch(Exception)
    {
      Console.WriteLine("A listener threw an exception in its handler");
    }
  }
}

Now when we run our test code, raising the events through the iterative handling routine, we get this

iteratedevent.png

As you can see, *all* of the EventHandlers ran.

Conclusion

We can't always control the code that other developers write (I'd argue we can't even really control our own code).  If you're writing code that's raising events to be consumed by any handlers (and most events I've seen are meant to be consumed) then you need to think defensively.  How can you protect your consumers from one another's bad behavior?  By coding defensively, that's how.  Simple patterns like calling GetInvocationList is a good defensive strategy.  It not only makes your own code more solid, it prevents those support calls from developers complaining that your assembly isn't raising events, even when they were the root cause.

Get the full source code here: InvocationTest.zip (27.37 KB)

OpenNETCF | Patterns and Practices
Thursday, May 27, 2010 2:59:54 PM (Central Daylight Time, UTC-05:00)  #     | 
# Tuesday, May 18, 2010
Why Windows Mobile Sucks - A Case Study

Let me lead off by saying I'm a bit of a luddite when it comes to phones. I like my phone to be able to make a call and occasionally send a text message. If I need to use a browser, I have a 30-inch Dell monitor and a PC that I can actually read web pages on. If I need to find a location, I have a nice Nuvi that doesn't require men to fiddle with a phone and that has a better interface that any other phone I've used. If I need to listen to music I have a Zune that lets me do that.

As far as phones go my favorite, other than that old StarTac that I still pine for, is my HTC Snap (also known as an HTC Ozone, Dash 3G and probably a few other names). Ironically it's also the target of my wrath over the last few days. As a user experience as far as a phone goes, it has everything I like, but if you need to actually create an application that does what you want on the thing, I think you're going to need divine intervention. And this, at its core, is why I think Windows Mobile, at least up through 6.5 just plain sucks.

My goal was simple - or so I thought. Create a simple application that collects GPS data at all times (yes, I know about battery life, blah, blah - let's assume that it's not an issue for this app). I know that a Smartphone never actually puts the processor fully to sleep and that even when the backlight/screen is off, I can still run code so this should be simple, right? Hardly.

Strike 1

The first thing I found, which was not a surprise, is that when the phone enters the "screen off" power mode, it shuts off the GPS receiver - or that's what I assume anyway. The GPSID driver stops giving me data anyway.

I know from cruising around the device's registry that the driver prefix is "GPD0:" and I know that WinMo is supposed to allow you to adjust power schemes through both API calls and some registry settings. "Supposed to" being the key here. I spent well over an hour going through and modifying the registry keys adding just about every combination I could think of along with soft resetting the device all with zero effect.

This is the first strike against this platform. Microsoft has documented that this is how the device is supposed to work, yet in a real-world scenario, it doesn't.

Strike 2

I know a bit about how Windows CE works, and my assumption is that these registry settings simply tell the Power Manager to set some driver device states under certain conditions. There should be no reason I can't do that manually, since I'm dealing with a pretty finite set of drivers (one) and states (two).

In order to manually alter the power state of a device, I need to know when the system itself changes state. For example, I need to know when the device goes into Unattended mode so I can then manually call one of the power management functions. To get notifications, we P/Invoke RequestPowerNotifications and pass it in a queue handle where it will send those notifications. Another hour of development later and I came to the conclusion that the device simply isn't sending out notifications for power state changes. I tested this in both C# and C to just be certain. Strike two.

Maybe I could just do something simple and just tell the Power Manager to never shut off the GPS. SetPowerRequirement should do that, but more testing proved that this, too, had no effect on this platform. After several hours of work and lots of frustrating tests that failed I was beginning to question the power manager altogether.

Strike 3

If the device wasn't going to tell me when it changes state, then I figured I'd just handle all of the state changes myself. I'd have the application periodically call SystemIdleTimerReset, implement my own application idle filter and dim the backlight myself. We're getting into some serious hack territory here, but some days you have to do what you have to do to solve the issue.

I implemented the timeout filter and used a call to SetDevicePower to shut off the backlight. The idea here is that I'd just never let the device go into the BacklightOff or Unattended mode and just turn off the backlight myself. I ran the test and ,surprise, my code had zero effect.   By zero effect I mean that the backlight shut off and the GPS turned off.

 I spent some time debugging to try to determine why I was able to shut off the backlight but not keep the GPS on and I finally realized that I was actually not turning off the backlight - the device still was. For the first time ever I was seeing a call to SystemIdleTimerReset succeed (i.e. return a success) but not  actually reset the timer. Strike three, and this was a huge one.

Conclusion

I continued to muck about with other APIs like PowerPolicyNotify to try to keep the backlight on and even some proprietary HTC calls I found online, all to no avail. I went back to basics and tried even the simplest of code in C and re-verified all that I had seen.The device appeared to completely ignore any call that had to do with the Power Manager while always giving me a successful return value.

All I can conclude from this is that HTC, in their infinite wisdom, decided to implement their own power manager to handle turning on/off all of these devices. and to make the WinMo platform itself happy, they just stubbed out all of the Power Manager calls to return success.

In the end, I told the customer that the phone they selected will never be able to do what they want and it looks like they're going to opt for another phone running a completely different OS.

This is a classic case of why I hate developing software the Windows Mobile platform, and one of the big reasons why the platform has been pretty much an abject failure. How can a software vendor even hope to develop an application that will work on multiple devices when the devices don't even follow Microsoft's published rules? The test matrix would be astronomically large and the costs, both in terms of the hardware you'd have to keep on hand and the time required to run the tests is simply prohibitive.

So is this HTC's fault? To a degree, sure. They should have followed the API's documented by Microsoft, but I still only place about 10% of the blame on them. Developers will, after all, always try to do things "better", and by better I mean that anything not designed in-house is obviously inferior.

But the real problem lies in Redmond. Every Windows Mobile device that ships gets a Windows Mobile logo. To get that logo they have to run the device through a set of Logo Certification tests (also called the LTK). The fact that a device can pass those tests with these obvious and fundamental variances shows that either the tests themselves have really poor coverage or the OEMs are getting waivers for these test failures.

Which it is doesn't matter because the end result is that Logo Testing is completely meaningless. The idea behind Logo Testing should be that when an ISV gets a device, they can depend on it to behave in a specific, predictable way. This device certainly doesn't. And this was just one device and just a small handful of APIs that I was attempting to use. Extrapolate that out to all of the device models that have been shipped and all of the APIs that any given application might call and you begin to see why this is so difficult (and very likely one of the reasons Windows Phone 7 is a complete redesign all the way back at to base Windows CE OS).

Personally, I don't do a whole lot of Windows Mobile development. I tend to focus on embedded Windows CE solutions where the hardware, OS and API set are fixed, finite, manageable and testable. When I see something wrong, we get the OS fixed - we don't path over the top of it (at least whenever we can). It's these little forays back into the Windows Mobile platform that reinforce and remind me of my utter distaste for it.

Hopefully Windows Phone 7 will have valid platform testing and a meaningful certification program but the reality is that I, and likely many, many other developers have been jaded by bad experience after bad experience with these and it's going to take a whole lot of work to gain our trust back. I'm not saying it's an insurmountable problem, but I certainly don't think Microsoft is going to get many more chances.

Compact Framework Code | OpenNETCF | Windows Phone 7
Tuesday, May 18, 2010 1:23:03 PM (Central Daylight Time, UTC-05:00)  #     | 
# Monday, May 17, 2010
New Microsoft Community Forums

Microsoft has finally declared the NNTP newsgroups dead.  It's where I've done my "community  service" for the past decade or so, so I'm a bit sorry to see them go.  They have started a set of new online Forums (which are live now) to replace these groups.

CE Device Development | Compact Framework Code | OpenNETCF | Windows CE Code
Monday, May 17, 2010 5:44:34 PM (Central Daylight Time, UTC-05:00)  #     | 
Getting the same MD5 hash from Native and Managed code

I'm working on a Windows CE project where we're storing an MD5 hash of a password in a SQL Compact database.  The solution is a mix of both native and managed code, and moth need to have access to both generating and verifying a password hash, so it's pretty important that we have functions for both that generate the same hash output given the same string input.  Here are the functions I'm using:

Managed

private static string MD5Hash(string str)
{
  MD5 hash = MD5.Create();
  byte[] h = hash.ComputeHash(Encoding.Unicode.GetBytes(str));

  StringBuilder sb = new StringBuilder();
  foreach (byte b in h)
  {
    sb.Append(b.ToString("x2"));
  }

  return sb.ToString();
}

Native

#include "wincrypt.h"

#define MD5LEN 16

DWORD MD5Hash(TCHAR *input, DWORD inLength, TCHAR *output, DWORD *outLength)
{
  HCRYPTPROV hContext = 0;
  HCRYPTHASH hHash = 0;
  DWORD status = 0;
  TCHAR hexDigits[] = _T("0123456789abcdef");
  DWORD hashLen = MD5LEN;
  BYTE hash[MD5LEN];
  TCHAR pair[2];

  if(*outLength < (MD5LEN + 1) * sizeof(TCHAR)) return ERROR_INSUFFICIENT_BUFFER;

  // Get handle to the crypto provider
  if (!CryptAcquireContext(&hContext,
                           NULL,
                           NULL,
                           PROV_RSA_FULL,
                           CRYPT_VERIFYCONTEXT))
  {
    status = GetLastError();
    return status;
  }

  if (!CryptCreateHash(hContext, CALG_MD5, 0, 0, &hHash))
  {
    status = GetLastError();
    CryptReleaseContext(hContext, 0);
    return status;
  }

  if (!CryptHashData(hHash, (BYTE*)input, inLength, 0))
  {
    status = GetLastError();
    CryptReleaseContext(hContext, 0);
    CryptDestroyHash(hHash);
    return status;
  }

  if (CryptGetHashParam(hHash, HP_HASHVAL, hash, &hashLen, 0))
  {
    memset(output, 0, *outLength);
    for (DWORD i = 0; i < hashLen; i++)
    {
      _stprintf(pair, _T("%c%c"), hexDigits[hash[i] >> 4], hexDigits[hash[i] & 0x0F]);
      _tcsncat(output, pair, 2);
    }

    *outLength = (_tcslen(output) + 1) * sizeof(TCHAR);
  }
  else
  {
    status = GetLastError();
  }

  DEBUGMSG(TRUE, (_T("MD5 hash of %s: %s\r\n"), input, output));

  CryptDestroyHash(hHash);
  CryptReleaseContext(hContext, 0);

  return status;
}

Both of these will probably work on the desktop as well, but I haven't actually tested there, so YMMV. 

This is also a great example of why I prefer C# for helping keep things simple.  Generally speaking, fewer lines of code means fewer bugs.

.NET Compact Framework | OpenNETCF | Windows CE Code
Monday, May 17, 2010 12:04:21 PM (Central Daylight Time, UTC-05:00)  #     | 
# Thursday, May 13, 2010
IoC for Silverlight Mobile?

Nick Randolph and I have been kicking around the idea of adapting the OpenNETCF.IoC project to Silverlight Mobile on Windows Phone 7 (and by "discussing" I mean I'm trying to convince him to do it for me).  I just noticed over on the IoC stats page that we've had some referrals from someone who apparently has either already done it or is in the process of adapting it for PRISM.  I've contacted him to see if we might collaborate, but this is exciting news. Having a dependency injection framework right out of the box for phone development can only help the community provide solid applications.

Dependency Injection | Inversion of Control | OpenNETCF | OpenNETCF.IoC | Windows Phone 7
Thursday, May 13, 2010 6:19:01 PM (Central Daylight Time, UTC-05:00)  #     |