Can you run python on an IoT device?
Imagine for a moment this, the ability to run Python on an IoT device, i.e. an embedded device such as a microcontroller. You might not have an appreciation for the problem, but developing for embedded systems is notoriously difficult. For our embedded developer friends, it means imagining not having to sift as much through datasheets and manuals. Imagine the amount of time, energy that teams would gain because of all that.
Last year, I read about micropython. It’s a library that allows a stripped down version of Python 3.x to run on constrained systems such as IoT devices. It took me a couple of minutes to understand the ramifications of such a feat. I am going to walk you through some of its advantages.
MicroPython is a lean and efficient implementation of the Python 3 programming language that includes a small subset of the Python standard library and is optimised to run on microcontrollers and in constrained environments … giving you a low-level Python operating system that can be used to control all kinds of electronic projects. — https://micropython.org/
Introduction
There is not argument about the fact that developing for embedded systems is notoriously difficult. Well, to be technically correct, I guess it is at least as difficult as any engineering project can be. But, in this particular context, many of the difficulties in the development of embedded systems come from dealing with the bare-metal hardware. This adds complexity to a the developer’s life, it requires him to sift through datasheets and manuals. And, then, he has to get everything to play nice with everything else. I am telling you, it’s not for the “faint of heart”.
Traditionally, embedded development make use of low level languages such as C. As such, they do not come with all the abstractions, features that higher level languages such as Python or Java have. This makes developing with C considerably more difficult to accomplish the same tasks.
The need for low-level control have made the use of low-level programming languages such as C the de-facto standard in the industry. As a consequence, out of the box, our developer friends do not have access to abstractions, or classes. They do not have access to many of the features that software running on servers and personal computers enjoy. I’m speaking of useful paradigms such as Object Orientation and Functional Programming. These have become industry standards and are not readily available on embedded and IoT devices. Our friend struggle their way without them and manage to built reliable, safe, critical, real-time systems. It is always a gigantic feat of engineering. And I take my hat off to all our embedded developer friends. You are unsung heroes.
However, on the other side, things are changing, our ability to fit more and more space and RAM into cheaper and cheaper constrained systems have increased. This has brought forward the emergence of sophisticated constructs such as abstraction layers,… These have empowered code reuse, and development speed up.
The difficulties mentioned earlier are only one aspect of the problems in embedded systems. Usually, and especially, in a manufacturing environment, there are stringent requirements on testing. This is to ensure stability and integrity. Practically, teams must spend a similar effort in developing a twin code base used for testing the actual hardware / IoT device. One code base for field application, one code base for production testing. The extra test code base will make sure that the microcontroller is 100% functional. It will make sure that it is properly integrated in the PCB where it seats. But, in total, that is two code bases, and that is a lot. — Two code bases to develop and maintain.
Good programmers write good code. Great programmers write no code.
Micropython: How does it help?
Well first of all, although it is a stripped down version of Python, it’s still very much Python. All a sudden, you have access to most of the features of the language. This includes the paradigms the language supports. Object Oriented and Classes, Functional Programming and Immutability, amongst others.
Usually, this means that a considerable effort shifts away from your team to, say, the manufacturer of the microcontroller. The manufacturer develops the abstraction layer whose API is micropython. He knows more about this piece of technology than your team does (hopefully). He is therefore better equipped to write and maintain this abstraction. For you and your team, this leads to a better and more robust system. Moreover, this abstraction layer is Open Source and shared across thousands of teams. They will report and work to fix issues, improve the quality of the abstraction. You and your team get all that richness, experience for free. You don’t need two code bases. This abstraction layer can emulate the entire microcontroller making it almost test ready.
The only other thing you then need, is a client for your test system and a set of functions that the test system will remotely invoke on the target IoT device (i.e the microcontroller)
Micropython: In a nutshell
If you look a the documentation
Micropython implements the entire Python 3.4 syntax (including exceptions, with, yield from, etc., and additionally async/await keywords from Python 3.5). The following core datatypes are provided: str (including basic Unicode support), bytes, bytearray, tuple, list, dict, set, frozenset, array.array, collections.namedtuple, classes and instances. Builtin modules include sys, time, and struct, etc. Select ports have support for _thread module (multithreading). Note that only a subset of Python 3 functionality is implemented for the data types and modules.
What you also get is a very useful interactive prompt over the serial port. This means that as soon as the board has power and you have access to its serial port, you can interact in real-time (yes, no compiling) with it, and its pins, and its peripherals, and its eeprom,… And for that, you can use any application. Any application that can send and receive packets on a serial port can interact with it. This means your test system can run any language in any framework.
Getting started
Getting started is extremely easy.
To get started one needs a board that runs micropython. Officially, the library promotes the PyBoard. But the documentation provide some other alternatives like the affordable and popular ESP32.
Then using any serial application (Arduino IDE, Putty, Docklight) open a serial communication session to the device comport, and you will get the interactive REPL prompt as shown above. From thereon, happy hacking.
Micropython: Advantages
I have already listed a few of the advantages that the library brings.
Now, it is true that this library is not a silver bullet. It might not be appropriate for any and all project requirements and characteristics. Yet, for a vast amount of electronic projects this should, at the very least, provide a fast, reliable and easy startup point and proof of concept.
The library caters for Python paradigms such as OOP and FP. It supports classes, mutable and immutable data types. It also implements generator, string manipulation, arbitrary precision integers. Closures, list comprehension, exception handling are more of the language features.
Micropython provides a fast, reliable and easy startup point and proof of concept.
Did I mention the absolutely, extremely useful interactive prompt over the serial port? I feel like I haven’t. Let me… It does feature an awesome, very useful, (dare I say, ground breaking?) interactive prompt over the serial port. And that makes debugging so much easier.
Test Systems And Test Executives
Test System development is definitely a good use case for micropython. Personally, I have been able to port this system to 4 different test products within a week. They run of STM32F405xxxx targets. And since then, I haven’t had to touch that code base ever again. This means no bug fixes, no new feature development,… In essence, limited maintenance requirements.
Now the only thing that’s left is a client for your test system. That’s one of the tasks I had to tackle while developing the test system mentioned earlier. I wrote a client for a Test Executive / Test System that will interact with micropython.
I have made that library public so that it can, in a similar way, speed up development on that front. The library is written in LabVIEW, to be optionally integrated with TestStand.
LabVIEW is a graphical programming language developed by National Instruments. It has a strong representation in Test Environments. And it seamlessly integrate with the Test Executive TestStand.
TestStand is also developed by National Instruments. It is in the same way widely adopted in the manufacturing industry as a Test Executive.
My hope is that you get as much fun as I did while working on this Test System. Using micropython makes your life easier on all fronts. This is especially true for the testing component of your project.
You can find the documentation for micropython here. The open sourced GitHub repository for the project is here. And the official website at https://micropython.org/
For the client library, you can find it in my repository here.
That’s it for me. I hope you enjoyed it. Leave a comment if you feel like it. I’m always happy to engage with the community.
From a fellow problem solver.
