Neil Cowburn - OpenNETCF - Poll Results: How Do You Test Your Smart Device Apps

April 7, 2005

Poll Results: How Do You Test Your Smart Device Apps

I previously asked about how you test your smart device apps. Only 4 people responded (thanks, guys). One could draw some rather dark conclusions from that, like:

Folks don't test their device apps (eep!)
People don't care enough to respond
Testing? What's that?

Of course, it could be true that only 4 people read my blog and I got a 100% response rate, but past experience of trying to illicit information from people shows that the response rate is usually < 10%.

Anyway, here is the concensus from those that responded:

Do you use the emulator or real devices?

One person said that only use real devices because the emulator is just too slow, even on fast hardware. Another said they used both for different purposes -- for accurate metrics, they opted for real devices, but for general testing, the emulator was sufficient.

Do you automate your testing? If so, how?

Two respondees admitted to using a shared codebase between desktop apps and device apps, so they used NUnit on the desktop apps for automated testing. One respondee reported using both automated and manual testing, while the fourth stated that device apps were manually tested.

What kind of metrics do you gather about your code?

One answered that cover coverage was the only metric they recorded. Another answered that they've gathered all kinds of metrics for all kinds of apps and have yet to find any that make real sense.

Do you run your assemblies through FxCop, NCover, etc? If so, how much benefit do you think this brings to your development process?

I'm going to quote one respondee here: "We use FxCop every couple of months to look for things that should be reworked". Another respondee (from the Symbian camp) had no knowledge of these tools.

What kinds of features would you love to see the .NET Compact Framework foster to improve testing (e.g. CLR profiling APIs)?

One vote for class-level granularity on memory pressure measurements.

Should the new testing features in VS 2005 support smart devices?

Two votes for yes.

There's just not enough data to draw any real conclusions from this poll. Thanks to those who (a) took the time to respond; and (b) showed interest in the results.

Comments [3]