How IoT is Changing Development, Testing & Software Release Processes

By Flux7 Labs
April 7, 2015



How does the Internet of Things (IoT) change the way we develop, test and release software? Always-on connectivity introduces a new set of risks and challenges for development and operations teams.

Last Tuesday, Flux7 CEO Aater Suleman (@futurechips) joined an online panel on this subject as part of Continuous Discussions (#c9d9), a series of community panels about agile, continuous delivery and DevOps. Other guest panelists were Shailesh Mangal, CTO of Zephyr, and Walter Buga, CEO of Arynga.

Continuous Discussions is a community initiative by Electric Cloud, which powers continuous delivery at businesses like SpaceX, Cisco, GE and E*TRADE by automating their build, test and deployment processes.

The discussion focused on the implementation and practical development of IoT) devices. The panelists were asked a number of questions about what they believed were the main changes that have occurred, as IoT has changed from being a theoretical concept to the point today where it is a practical reality.

The Essence of IoT

The first question that the panellists were asked to ponder on was what actually is the essence of IoT? The panelists had varying views on where the focus of IoT is. To Aater “the essence is machine to machine communication, having these devices connect to each other over the Internet. I have built IoT device networks that did not involve the cloud, before the cloud actually existed, and when the amount of data was not that large, so it was not necessarily a data play.”

Velocity, Quality and Features

The panelists were then asked what they felt was most important in relation to IoT: velocity, quality or features.

Aater felt that this very much depended on the particular IoT market segment you were focusing on. The needs of IoT devices aimed at consumers are very different from the needs of businesses.

He gave an example of a toy that uploads data. In that case, features are very important, and you will test to see how well the features are working. You could, for instance, test to see which buttons the user are actually playing with. Quality is less important for this type of consumer good because if the product fails, it can simply be taken back to the store.

It is different with business applications, however. Quality is definitely more important here.  

“If you’re sending something to Mars, it had better work. Failure is not an option.”

Aater expanded on this theme: “We have large enterprise customers and small clients; the startups are the playground where we try the latest and greatest of DevOps, and as we have confidence we move things up to the Fortune 500 and 1000 level. Likewise, velocity will be high for the consumer market and lower for the enterprise market with quality being much more important.”Testing in an IoT World

The next question posed for the panellists was, how does testing change in an IoT world?

Aater believes that with IoT, there is quite a change in viewpoint when it comes to testing. “It’s a new way of connecting to the Internet, a new interface to the Internet, which we use to connect to from our personal computers and smartphones. The ground principles do stay the same: test your back end, test your front end – but your front end may be a device with no monitor, you may have no I/O on your device, you may need backdoors and ways of interacting with the device with no physical access. It’s definitely very different.” He believes that performing QA with IoT solutions is somewhere in the middle between how you would perform it for software and for hardware.

One similarity he sees between testing hardware devices and testing on IoT systems is the need to incorporate Design for Testing (DFT), which permits automatic testing. It is also important to recognize that in reality there will be blockages to internet connectivity, and this has to be allowed for.

Software Release and Delivery with  IoT

How does software release and delivery change for IoT? How do you coordinate and orchestrate releases?

All the discussion participants agree that  IoT has totally changed the way that products are delivered to the public. The biggest change is the need for the software in devices to be updated, preferably without user intervention. An issue with upgrading systems for devices like the iPhone at the moment is that the user chooses whether they will upgrade their system or not, and when it happens. The reality is that approach will change with IoT devices. They need a more consistent update procedure, particularly for items which would have real safety or security issues if the updating was left to the consumer. As one panelist commented, imagine waiting until a customer decided to undertake the upgrade on the braking systems in their car. Of course, it is important that the system does not decide to upgrade the braking software when you are driving along!

According to AaterWhen we are doing continuous delivery and deployment with these devices, the velocity is very different.You can have a thousand deployments per day, and I’m a big fan of that.”

He does recognize that not everybody will be able to upgrade their software at the same point, so the system needs to be designed around this. It is important that an upgrade is not pushed until the network is robust enough to take it. He gave an example: “a solution I’ve seen working in the field well is to design the system so that people can be left behind, and you can recover from that. For example, we have a client who is a large manufacturer of moving machinery. They don’t push an update until the network obeys a certain level of robustness and testing. If the device does not go on the network they won’t try to push an update.”


The prevalence of connected devices is already changing the way businesses look at and plan for IoT data, and systems to support IoT must be looked at with renewed perspective as well. Effective infrastructure is essential for successful IoT programs, but so are the development processes that infrastructure must support. IoT demands new thinking that considers strategies for high scalability architectures and integration of diverse technologies and systems. Continuous delivery is becoming the norm, and IoT businesses will need to approach this model with thoughtful consideration to plan ahead for a wide variety of possible circumstances.

We look forward to more conversations on this and related topics and thank Electric Cloud for the opportunity to participate in this panel.