Internet Of Things (IoT) is a very important buzzword now & hardly anyone is not aware of the terms like Smart Wearable devices, Smart Thermostat, Smart Car, Smart Home, Smart City, etc. With new advancements in device-to-device or device-to-server communications, Performance Testing tools are evolving to add right protocols & techniques in their test suites to validate the IoT system performance characteristics.

<You can write to support@qaelitesouls.com to receive our free e-booklet on IOT Performance Testing>

What is IoT?

IoT is a network of physical objects (like devices, automobiles, home appliances, etc) that has sensors & use various embedded technologies to collect data from environment for intelligent decision making. It is a rapidly expanding technology revolutionizing the information collection & processing to make quick and intelligent decisions. Smart devices are not just meant for sending useful information; it can also analyze & make decisions based on the programmed instructions.

According to Gartner, there will be around 26 billion IoT devices in use by 2020. According to McKinsey Global Institute, IoT technologies will impact economy by $6.2 trillion by the year 2025. This clearly shows there will be a huge demand for the way these smart devices can help different industrial sectors & in order to cope with this demand, there needs to be enormous change expected in development & testing methodologies too.

Why IOT systems demand new set of Protocols?

The physical devices involved in IoT systems, must communicate with each other (Device-2-Device), the data collected by the device must need to be sent to the server infrastructure (Device-2-Server) and the server infrastructure must need to share the processed data (Server-2-Server), to web services & other business applications and sometimes back to the devices.

In order to facilitate above mentioned communications IoT systems requires new set of protocols for communication. These new communication channels are not very prevalent in our experience using web & mobile applications.

Most of the IoT protocols use a TCP or UDP transport layer for communication. There are millions of smart devices used in IoT systems, with each device using custom or standard protocols for their communications. Hence, one of the biggest challenges with the IoT is the lack of standardization.

Some of the prevalent IoT protocols include MQTT, CoAP, AMQP, XMPP, DDS, REST APIs over HTTP, etc.

IoT Performance Testing Tool Choice

The Performance Testing tools that claims to have support for IoT protocols needs to include atleast the key protocols like REST API over HTTP , MQTT & CoAP. If the IoT application under test uses only REST APIs over HTTP for communication, then performance testing tool which support this protocol would suffice.

A robust performance testing tool which supports all IoT system protocols are not yet ready. There are many commercial testing tool providers who claim they support IoT system performance testing, if they offer REST API over HTTP protocol testing features. So, the choice of the performance testing tool should be based on its support for IoT protocol used by the IoT application under test.

SmartBear claims its support for IoT protocols, with its plug-in releases for MQTT & CoAP protocols added to their Ready API suite.  Ready API is an end-to-end API readiness platform that combines the power of SOAPUI Pro, Load UI Pro, Service Virtualization & API Monitoring solutions.

Several performance testing tools like Apica, Neoload, etc claim their support for IoT protocols. I would highly recommend carrying out a tool evaluation study to check the feasibility of the tool to support the IoT application under test. Not all IoT applications are built on new set of IoT protocols that are very different from web & mobile applications.

Also, it’s high time for the tool vendors to bring in some new techniques to performance test the applications instead of sugar coating on the traditional techniques that have been followed for more than 2 decades.

Top 5 considerations in IoT Performance Test Strategy

Appropriate network sniffing tools can be used to capture the network traffic & thus recorded requests can be crafted manually on the performance testing tool, which supports IoT protocols for carrying out load tests.  But the performance testing tool should support the underlying protocol used by the IoT application under test.

Performance Testers need to be aware of new dimensions introduced by IoT applications that are not available in web and mobile applications. The IoT application performance test strategy should focus on below mentioned areas.

• User Interface Performance Tests
• Network Performance Tests
• Internal Processing Performance Tests
• Longevity tests
• High Data Volumes Tests

Top 10 Performance Testing Challenges

Here are the top 10 challenges faced during IOT performance testing.

• Choosing right performance testing tool
• Diversified Protocol Stack
• No Standardization for Devices & its performance evaluation techniques
• Realistic Workload Model Creation
• Huge investment in tool license cost for cloud based load generation
• Need for long duration endurance tests to certify stability
• Need for Sophisticated Network emulation tools
• Test Data Management strategies for creating huge data volumes
• Complexities involved in setting up a dedicated Performance Test Environment
• Need for investments in custom tool / utility development

Due to unavailability of sophisticated performance testing tools & proven techniques, IoT performance testing is considered to have several challenges as of now. As there are not much standardization seen in IoT protocols used & due to the diversity of devices in IoT platform, performance testing tool vendors also have challenges in coming up with robust solution. But still there are several work arounds are currently used to carry out performance analysis in bits & pieces to assess the IoT application performance. Similar to mobile application performance testing, it’s just a matter of time & we should soon get more sophisticated tools available in the market to intelligently assess the IoT application performance smartly.

You can write to support@qaelitesouls.com to receive our free ebooklet on IOT Performance Testing which has the below sections

• Introduction 
• What is IOT?
• Key Components of IoT Platform
• How IoT Platform works?
• Why IoT Systems demand new set of Protocols
• Prevalent IoT protocols
• Need for choosing right Performance Testing Tool
• Detailed view on Performance Test Strategy considerations
• Detailed view on Performance Testing Challenges
• Conclusion