Embedded systems are now more intelligent, more interconnected, and, of course, more complex than before. To keep embedded systems robust and as close to error-free as possible, development teams need effective methods for testing to verify that the system functions as expected. The most critical, and often the most challenging, aspect of testing is running the software on the microcontroller.
This article will introduce developers to how to use these new technologies, as well as the equipment and tools required for their implementation.
Changes in Embedded Testing Requirements
Modern embedded system testing tools require developers to use the following four main components to fully test their systems:
Debugger that supports tracing functionality
Communication adapter/sniffer
Logic Analyzer
Analog-to-digital converter (ADC)
Using these four components, developers can test embedded software at the system and microcontroller levels, and also delve down to the instructions executed by the microprocessor.
In today's development environment, this is crucial to ensuring that the systems built not only meet requirements but also operate reliably.
At first glance, it may resemble a very traditional embedded system testing tool, but its advancements and new features truly stem from a completely new way of analyzing how microcontrollers are functioning, called "deep insight analytics".
Using in-depth insight analysis to test software
Deep insights analytics enable developers to analyze systems at runtime. Deep insights analytics has three key components:
RTOS-aware debugging
Runtime analysis
Analysis and code coverage
Developers typically use basic debugging techniques (such as breakpoint debugging) to try and understand how the system works after designing the application, and then immediately begin testing. Testing with breakpoints present is only superficial and does not allow developers to truly understand how the microcontroller is operating.
With deep insights analytics, developers can delve deeper into RTOS, runtime behavior, execution analytics, and coverage beyond basic testing and debugging.
To add in-depth insights and analytics to testing tools, developers must use specialized debugging tools such as Segger Microcontroller Systems, J-Trace, or J-Link Ultra+.
J-Link Ultra+ uses standard JTAG or SWD interfaces to extract trace data from the onboard debug module. This information can be used to perform various analyses, such as RTOS-aware debugging.
With RTOS-aware debugging, developers can monitor task execution as test cases are run. For example, developers can gain deeper insights into the following:
Maximum stack usage
Task run count
Task status
This shows a sample test session using Segger's embOSRTOS and EmbeddedStudio. This view allows developers to gain insight into how the RTOS is functioning, but it does not provide complete information about how the application is performing.
Developers can further enhance their testing tools using Segger's free SystemView utility or Percepio's Tracealyzer tool. These tools provide developers with runtime analytics, allowing them to visually view and analyze their application's performance as they run test suites.
To collect this tracing data, developers need to configure their applications to perform tracing within their IDE or use configuration tools. If no tools are used, they can manually integrate low-level libraries.
Runtime analytics can provide developers with a wealth of information about how their application is running. For example, developers can:
Tracking the time and sequence of events
Get the maximum, minimum, and average execution time.
Visually view task execution and when tasks switch.
Monitor CPU load
Analyze task statistics
Identify potential problems such as priority inversion, task thrashing, and deadlock.
While RTOS-aware debugging and runtime analysis capabilities have been added to testing tools, this may not be sufficient. In many instances, errors may be hidden in code that is never executed during the test. For developers, understanding which lines of code have been executed can be very difficult. Tools like J-Trace offer significant convenience in this regard.
How to track executed code
J-Trace uses the Embedded Trace Macrocell (ETM) port in the microcontroller to perform instruction tracing. Through instruction tracing, J-Trace can "see" every CPU instruction being executed on the processor, as well as the exact path of the code.
Using this analysis in testing tools, developers can determine whether test cases achieve 80%, 90%, or 100% code coverage.
If test coverage is only 95%, but 100% coverage is required to deliver the product, they can use a free utility similar to Ozone to see which lines of code have been executed, and more importantly, which lines of code have not yet been executed.
You can then add new test cases to ensure that these missing lines of code are executed in the tests.
With the powerful tools provided by deep insights analytics, developers are now free to focus on the other components necessary to build effective testing tools. Another crucial component of a testing tool is the tool used to communicate with and control the behavior of the microcontroller during the testing process.
Command and control embedded systems
Each embedded system has different requirements for how it interacts with the outside world. Some devices may communicate via a simple UART, while others may use CAN or TCP/IP.
In order to successfully build test tools that can communicate with the system and execute commands, developers must add communication hardware and software to the test tools.
Although embedded systems use a wide variety of communication interfaces, one interface is more commonly used than any other: UART. Embedded software developers are accustomed to using UART, and including this interface in test tools is important for several reasons, including:
For debugging purposes, such as printing messages (although these messages should be sent via the debugger).
The device issues a command
Monitoring internal communication between multiple devices
Easy to use
One universal UART tool that every developer should always have on hand in the lab is the BOB-12731FT232R USB-to-UART evaluation board provided by SparkFunElectronics.
These inexpensive evaluation boards can be easily connected to any embedded system and function as a simple communication port on a PC. They require no special drivers or software to communicate with the embedded system. Developers simply need to open the COM port to begin sending and receiving test messages.
Verify each logical state
To fully test an embedded system, developers need to verify the internal workings of the microcontroller and the external logic it generates. This logic could be simple input and output states, as well as low-level communication such as I2C or SPI.
However, if developers use analog-to-digital converter (ADC) and digital-to-analog converter (DAC) boards, monitoring input/output status and low-level communication can be costly. Developers can utilize several techniques to reduce costs while monitoring these signals and improving test functionality.
The first tip is to use a development board from the microcontroller on the system to monitor the microcontroller pins. For example, if developers are using STMicroelectronics STM32F767 or STM32L4 microcontrollers, they should first purchase an STM32F767Nucleo board or an STM32L476RGT6Nucleo board.
Then, they should use the pin headers on the board to directly jump each I/O pin to the corresponding pin on the system. Since they have already developed low-level drivers for the microcontroller, these drivers can be easily tuned to monitor the microcontroller's input and output status.
Developers can also add a small amount of additional code, such as a USB driver, so that the development board can be directly plugged into the test host.
USB can be used to receive input/output sample status data, which can then be correlated with commands issued and received from the system to determine whether the operation is fully in line with expectations.
In addition to using a development board, testing tools can also use logic probes, such as LogicPro8, which is also from SparkFun.
These logic analyzers are versatile, allowing each input to be modified via software for the object being monitored.
For example, developers can configure the first two inputs to monitor the switch input, use the next two inputs to monitor I2C communication, and use the remaining inputs to monitor SPI. Data can be easily acquired and then synchronized with the remaining components of the testing tool, providing a comprehensive understanding of how the embedded system is operating.
Tips and tricks for building testing tools
Clearly, testing tools are essential for many modern embedded systems. While obtaining certification can sometimes be extremely difficult, the returns and improved system robustness that certification brings can outweigh the initial investment.
When building testing tools for the first time, and even when upgrading them, developers can utilize several techniques to ensure the most effective testing tools are built. These include:
Use a development kit with the same processor as the primary target to monitor the microcontroller's digital inputs and outputs.
Invest in debuggers that support tracing and leverage free software packages for the deepest possible understanding of system operation.
When running software tracing, always use worst-case testing to ensure you capture the worst-case scenario.
If you don't have enough funds to build a complete testing tool, start by building a smaller one and then refine it over time. Even performing only partial testing is better than not testing at all.
Spend the necessary time mastering the different tools and components to be used in the testing tools.
Don't be afraid to build your own interfaces; make full use of existing software to monitor system operation.
Don't make any assumptions! If you don't monitor the output or trigger the input, you're likely to let errors slip in.
Summarize
Developing test tools for embedded systems is a low-cost method to improve their reliability. Carefully selecting the components of the test tools allows developers to easily monitor the external behavior of the software.
The most critical yet often overlooked aspect of testing is checking the tracking data, which is now easily accessible from the microcontroller.
Using this tracking data, developers are able to perform in-depth insights and analytics while executing their test cases to ensure that their software works as expected, even for individual instructions.
