We’ve all been in a meeting, conference or seminar where the talk goes something along these lines:
“We collect the data at the device, then we send it up to The Cloud where we can do analysis and other things on it.”
This is usually accompanies by the vague graphic of a line – sometimes solid, sometimes dotted – pointing to the generic PowerPoint cloud image.
I hate this statement. I hate this cloud. This is usually a huge red flag that the speaker or author really has no idea what they’re talking about. It a sales or marketing cop-out that is the technical equivalent of the popular “then a miracle happens” cartoon.
What does “send it to The Cloud” even mean? What cloud? How is the data going to get there? How will I get it back out? I’ve got other questions about “the cloud” that I’ll address in another post, but these three are the biggies that you should always raise your hand and ask.
1. What is “The Cloud?”
Here’s the little secret – there is no single “cloud”. There are a boatload of them, and the list seems to be continually growing as it gets cheaper to stand up servers with a lot of storage and as more and more use-cases make use of off-device storage. “The Cloud” can mean a whole panoply of things, including (but definitely not limited to):
- An on-site data store such as SQL Server, Oracle, MySQL or the like
- A third-party hosted storage solution, which might be any one of these:
Again there are plenty of others, these are just most of the ones I’ve dealt with in the past year or so.
And bear in mind that not all customers are going to be amenable to all clouds. Some customers aren’t so comfortable putting their data on servers they don’t control. Some aren’t comfortable putting their data on servers in particular countries where governmental agencies are known to mine them. Some customers simply have predispositions to different services for different reasons.
Maybe they like Azure because it provides a simple interface for their .NET enterprise apps. Maybe they like Amazon’s scale. Maybe they like Brand X just because they do. The point is that if you have more than one customer, you’re probably going to need to look at more than one cloud provider. We’ve got customers that use multiple clouds due to the benefits each provides for certain types of data, data retention policies or total cost of use.
2. How do I get my data into “The Cloud”?
Yeeaaahhh, about that…since there are a boatload of cloud services, there are a boatload of ways that an app gets data into them. Network databases might use ODBC or a proprietary/specific API set. Services on the web typically use a web service interface. Maybe REST, maybe OData, maybe something else. The point here is none of them are the same. None of them.
So that means that if you have to support multiple clouds, and you will have to support multiple API sets. Of course you can abstract them all back to a common interface – I’d certainly recommend it, it’s what we do using the OpenNETCF ORM - but there’s still work to be done to actually implement each.
3. How do I get my data back out of “The Cloud”?
Just like putting the data in, getting it out requires an API, and again, they are all different. Another thing to consider in the “getting it out” part of the equation is how you actually use it.
Some clouds allow you to run services and/or applications on the servers as well and data access is direct. Sometimes you have to pull it out back to another server. Once again, this means more work from a development perspective. And again, you’ve got to multiply that by the clouds you need to support.
And how about data retention? Some clouds are not “forever” storage. The data gets purged on a time basis. If you need to keep the data archived for later data mining then add that work onto your plate too.
So the next time you see that slide with the cloud on it and the speaker says “just send the data to the cloud” raise your hand. Take them to task over it. We build software, and while that seems like magic to some, we don’t do miracles.