An Introduction for Firmata and its use with Test Systems
I recently wrote about micropython, a library that allows python 3.x to run on constrained / embedded systems. This was particularly interesting with regards to test systems. As a matter of fact, using the library has reduced the development of test code for embedded devices. This is particularly helpful in a manufacturing / production environment. I even presented a client library for LabVIEW and TestStand, since these two are particularly prevalent in this industry.
Today, I want to introduce a different library, Firmata. This library is, very much, a different beast from the previous one. But, just as micropython, it also aims to provide an abstraction for the target it is running on. It enables the reuse of the same interfaces, clients and APIs to interact with a wide variety of different embedded systems. A test system can interact with any embedded device that implements the library. In fact, any PC/Laptop can interact with devices that are running with the Firmata protocol.
Firmata: What is it?
Firmata is a generic protocol for communicating with microcontrollers from software on a host computer. It is intended to work with any host computer software package. …The aim is to allow people to completely control the Arduino from software on the host computer
Let’s talk a little bit about the problem this library tries to solve.
Say, your team is developing a new product that consists of an actual PCB and a MCU running on it. From experience, your team will spend an appreciable amount of time bringing up the different peripherals of the board. Once they are all up and running, you will then develop the field application as per requirements.
In parallel, your test team will spend a similar amount of time bringing up these peripherals. This is so they can develop a test application. The test application is used during the manufacture of this board. It allows test systems to assess whether the board and its different peripherals are functional. It will test the different features of your product against a set of specifications.
It is quite a lot of code and time spent on this. Firmata proposes to considerably reduce that:
- For the field application, Firmata can be used as a rapid proof of concept. This allows quick validation of ideas, hardware, specifications and requirements for the project.
- For the test application, Firmata can be used as is. This is possible since it exposes much of the hardware functionality over a communication link. This link can be a serial communication port for example.
Firmata is a protocol for communicating with microcontrollers from software on a host computer. The protocol can be implemented in firmware on any microcontroller architecture as well as software on any host computer software package.
Getting Started
How easy is it to get started? Well, it is very easy. You need:
- A embedded device running Firmata. To start, any arduino will do fine.
- A communication link to your Laptop / PC. The easiest is the standard serial communication link over USB. Since it is the available with all Arduino boards.
3. A Client for the laptop / PC.
- On a windows system, open Microsoft Store. Search and install “Windows Remote Arduino Experience”. It’s a piece of software written by Microsoft. It makes use of Firmata to allow your PC to interact with the outside world.
- Or, you can choose and download a client from a list spanning many programming languages. There are quite a few to choose from. Python, .NET, JavaScript, Clojure, GO, … A list of client implementation (and languages) is provided here. Whatever language you choose, there is probably going to be an implementation. The client will typically come with a public GitHub repository and a “how to get started” guide.
Firmata: High Level Overview
The number one feature of the library that made me fall in love with it is its abstraction of the transport layer. In other world, you can use a serial communication link, or Bluetooth or WiFi,… And the project is very much alive. It is open for growth and there are new proposals to expand its functionality.
But, let’s take a step back and look at the big picture.
If you look a little bit closely at what the Arduino project is, you will see that it is:
An open source and extensible project.
It is an abstraction layer for your board hardware. And it does a very good job at doing that - as seen by its popularity. You are not directly interacting with the low level components of your target. Its quirks, peculiarities are hidden, abstracted away, behind a common interface. This makes it possible to use the same sketch/arduino code on different hardware with some minimal customization.
Arduino is an open-source electronics platform …
Open source and extensible software — The Arduino software is published as open source tools, available for extension by experienced programmers. …
Open source and extensible hardware — The plans of the Arduino boards are published under a Creative Commons license, so experienced circuit designers can make their own version of the module, extending it and improving it….
What the quote above doesn’t make too clear (at least not for me), is that you can develop and configure any board to run with Arduino. You wouldn’t be incorrect in assuming that the Arduino project is more a specification than a particular product. In other words, you can adapt any board to integrate the Arduino ecosystem.
Most Arduino boards consist of an Atmel 8-bit AVR microcontroller. Typically, ATmega8, ATmega168, ATmega328, ATmega1280, or ATmega2560. But you can adapt other micro-controllers to run with Arduino (see here for a tutorial). You can also adapt a completely different family of controllers altogether so it integrate in the Arduino ecosystem. (See the official project for STM32)
How is all of that relevant for Firmata? Well, what Firmata does, is that it builds another abstraction layer on top of the Arduino Implementation. Arduino is a mature technology involved in many projects across various industries, expertise, use cases. The Firmata project builds upon all of that maturity.
In practice, as in my personal case, this previous paragraphs mean that if you have a custom board (mine was running STM32 hosts):
- You want to find an implementation of Arduino for your host. Hopefully, your manufacturer has already done that.
2. Then adapt the Firmata layer to run on top of it.
It can sound like a very arduous thing to do, but it’s not. One thing that I have noticed though is that documentation of the process is seriously lacking. Tutorials that walk you through the process are rare, and I’m planning to write on that nearly.
One other architectural aspect of the protocol, that I absolutely love, is that it aims at abstracting the transport method. This allows you to reuse the same client (running on your laptop) over different medias. Serial Communication, WiFi, Ethernet, Bluetooth are all supported possibilities. The only requirement really is that there is a communication link between both devices.
Firmata: Features
As I explained previously:
Firmata exposes the internal peripherals of your target device to the client.
Some of the modes or functionality that can be accessed and interacted with:
- Digital Inputs, Digital Outputs, Analog Inputs, Input Pull-ups.
- PWM, Servo and Tone: All these modes are flavors of PWM. The servo is specialized to control servo motors. While the tone is used with a speaker or piezo buzzer to generate sounds. This latter is an advanced mode of the library.
- Shift: This mode enables pushing and retrieving data from shift registers on the device.
- I2C: The client can remotely send and retrieve data from devices on the I2C bus on the target board.
- One-Wire: This protocol is very similar to I2C - Only one wire (and another for ground) is used to communicate between the (one) master and the other (N slaves) devices on the bus.
- Stepper: Specialized protocol to control stepper motors.
- Encoder: Encoders are sensors that generate digital signals in response to (linear or rotational) movements. These signals can convey information such as speed or position. This mode allows the client to obtain that information.
- Serial: Exposes UARTs (up to 4) on the remote device to the client on the laptop / PC.
Below are some advanced modes:
- SPI: Sending and retrieving data from devices on the SPI bus on the target board.
- Sonar: Interacts with a “sonar” sensor to obtain distance information about some obstacle situated in the “vicinity” of the sensor.
- Tone: see PWM above
- DHT: To interact with a “Digital-output relative humidity and temperature” sensor/module
Relevance for Test Systems
As I explained earlier, the library is particularly interesting to a test system.
- It can cut, by a factor of 2 or 3, the time your test team is spending to put together test code for embedded products.
- In the long run, it limits the risks issues related to maintainability and bugs. This is possible because the library can be considered a mature one.
- Company-wise, it enables reuse of client “test codebases” across different projects… If they all use Firmata. The library can even be extended to support specialized forms of transports, if necessary.
A Firmata Client in LabVIEW and TestStand
My initial use case involved the implementation of a test system with TestStand from National Instruments as a Test Executive and LabVIEW as the primary programming language. However, at the time, there was no client implemented in LabVIEW. As a result, I developed one to address that shortcoming. This client implementation is made publicly available here.
The next sections will give an overview of this client and its integration with a Test Executive such as TestStand.
The client is written in LabVIEW.
LabVIEW is a Graphical Programming Language developed by National Instruments.
Architecturally, this project decouples transport layer from the client interacting with the embedded device. This is follows pretty much a flavor of “micro-service”. In this setting, each of these transport “micro-service” is addressable via the uniquely named queue associated with it. This allows multiple devices to be available at the same time. It then also enables one client to decide, on the fly, at run-time, which device to interact with. The appropriate queue can then be chosen to transmit / receive messages.
The main messages, as specified by the protocol, are implemented as a set of API functions / Public VIs. These will interact directly with the remote object via the associated queue.
The transport layer also offers the ability to push data to a UI Service with, possibly, state persistence.
Integrating with the Test Executive
In the case of TestStand, the API functions / Public VIs provided should be enough. An example sequence file is included in the GitHub repository.
The relevant transport VIs must be separately initialized beforehand and cleaned-up afterward. A transport VI will accept the name of its queue as an input argument. The queue name can be used later on to interact with the associated remote object.
From thereon, the API / Public VIs can be called with the correct queue name.
That’s it… The intention is to convey that, especially in the case of a Test System, Firmata can bring huge benefits on the table. It will standardize test code bases across a company and its project. And in the process, the projects will benefit from a mature and stable test library. This will increase the quality of the overall test systems and products.
That’s it for me. I hope you enjoyed it. Leave a comment if you feel like it. I’m always happy to engage with the community.
From a fellow problem solver.
