Scaling the Internet of Things
The Internet of Things (IoT) is here. More often, we are hearing about new home automation platforms and connected devices coming to market from smart watches, thermostats, home appliances, cars to smart meters. These devices are not only connected, but intelligently communicating to other devices and data feeds, and collecting huge amounts of data.
“As IoT evolves from its nascent stages, the biggest challenge is to scale the IoT architecture in the companies, to meet the future demands”
Analyst firm, Gartner predicts that we will have 26 billion devices connected to the internet by 2020. The companies playing in the IoT space are constantly thinking about this growth. As IoT evolves from its nascent stages, the biggest challenge will be scaling the IoT architecture. Today the process can seem overwhelming. There are too many options and solutions to choose from, and at the same time there are only a handful of mature tools available in the market. Fortunately, we can learn from companies that are already planning or undertaking this task at hand.
To start building architecture that will meet the future demands of IoT, you will want to consider these six areas as you plan your initiatives:
1. The Cloud vs. Local
There are advantages and disadvantages for cloud and local capabilities depending on your use case. You don’t have to opt for just one solution. Most companies are now setting up hybrid solutions. For example, if you require short-term intermittent connectivity, optimize local for cheap sensors with limited storage and compute. You can also use local for simple command and control, and store and forward architecture, and default, safe behavior.
The cloud definitely has its advantages such as cost-effectiveness, flexibility and accessibility. In IoT, data is collected locally and brought to the cloud where data feeds coming from various sources are managed, analyzed and then actions can be taken. For us that means, applying analytics and machine learning algorithms to bring intelligent, automated and tailored home energy services to consumers. However, there can also be difficulties when scaling. Cloud latency issues can occur with IoT-based applications with evergrowing data sets and unreliable home Internet connections. One of the key concerns in such an environment is having an architecture that is designed for failures, both at local level and in the cloud. This is what differentiates a scalable architecture from one that is unlikely to meet the reliability expectations of IoT.
Security is probably the top issue in IoT. Questions are being asked about what is collected and who has an access to that information. All the basic considerations of information security apply such as authentication, authorization, audit trails and encryption of data. One key area to focus on is, protecting remote access as it is the most critical attack vector for a large scale disruption. If someone has physical access to the device, there is not much one can do, so have reasonable physical access safeguards but don’t over-engineer this area as it is very difficult to thwart a persistent attacker.
Cloud storage security can be a drawback for businesses with highly sensitive data. The scandals involving the NSA and more recently, the widespread Heartbleed Open SSL bug, has called into question the security and privacy benefits that these cloud services offer. For consumers, the ability of third parties to gain access to personal data with or without the knowledge or cooperation of service providers is frightening. That said, many are still willing to look the other way for the sake of convenience. At the end of the day, anything you put out on the Internet, whether it be in cloud or social networking cannot guaranteed to be 100 percent safe, however, proper risk mitigation strategies can be employed to minimize those exposures.
3. “Always On” or “Sometimes On”
Whether the data needs to be processed in real-time or can be processed in batches? This is an important area to think about as not all use-cases require always on connectivity. Think of time-sensitive actions that need to happen right away, versus actions that can be delayed without any significant loss in functionality. For example, firmware upgrades are occasional while remote commands need to be done in real-time.
Another key consideration in an always on architecture is prioritization. Borrowing from the networking domain, IoT architecture will need to evolve to put in proper prioritization and retransmission capabilities to make sure the most latency sensitive actions or data get priority over congested and unreliable home Internet connections.
4. Two-Way Communication or Data Collection Only
Are you doing things in real-time? Many smart IoT devices offer two-way communication as it is the foundation of the IoT value stack. It allows for control and automation. Many users prefer to have things done for them instead of getting recommendations on what to do. In home energy, though some consumers are more proactive than others, we believe that the majority prefer the convenience of the automated approach which requires two-way communication architecture. Most consumers don’t want to continually manage and make adjustments on the mobile phone or thermostat to save energy. That’s how user-convenience services likes ours differentiate.
5. Automation or Recommendations
As mentioned above, we believe that users are inundated with recommendations so we take the approach of automation - what can we do on behalf of the user, while ultimately giving the user a choice. If automation is the path you choose, some of the issues to think about are: transparency and control. Users want to know what you are doing with their data and how you are going to use it, when you are doing it and why. Also, it’s important for users to have control when they want it. We provide that choice for consumers, allowing them to override our settings. We also learn from those overrides and respond intelligently with further adjustments based on those actions.
6. Big Data Considerations
There are too many Big Data platforms available to choose from. The key consideration for picking the right tools is carefully defining your core use cases. It’s easy to say you need a real-time system but most applications can operate with a batch-oriented system. The former requires a lot more sophistication and deeper skillset on the team so choose carefully. One strategy is to start with the simple batch-oriented approach first, learn from it and then perhaps evolve your approach over time if you really need real-time access.
One of the often overlooked consideration is building a system and application that can deal with data quality issues and data holes that are likely to arise when you are dealing with data, originating from a distributed set of devices, running different versions of firmware over an unreliable set of communication networks.
Another consideration is thinking about data aging and archiving strategy. Not all data is created equal and putting in place processes to keep the most relevant data on faster storage while archiving data needed occasionally on slower storage. This will help put in place safeguards to keep your data costs under control as the data grows.
From our experience, it’s important to take an agile approach to the problem and to tackle the most critical problems first. It’s also important to remember to plan for learning with the first couple of projects because plans evolve, mistakes will be made and new technologies, tools and methods will come to your rescue.