Artificial intelligence → Module 4 → TinyML

TinyML: artificial intelligence on small boards

In the previous lessons small models – a tree and k-NN – ran on the ESP32 and micro:bit. But can a neural network that recognizes words or images also fit on a board? This field is called TinyML (from tiny and machine learning). In this lesson we calculate how much “intelligence” fits into a small chip and get to know the tools that are used.

How much memory our chips have

A laptop has hundreds of thousands of times more memory than a micro:bit. That is why models are trained on a computer, and only finished, “shrunk” models are moved to a small chip.

Let's calculate: how big is a network

Each connection between two neurons has a weight, and each neuron also has a threshold. For layers with a and b neurons that is a · b + b numbers. A decimal number (float) takes 4 bytes; if we “shrink” it to a whole number from –128 to 127 (int8), it takes 1 byte.

# How much memory does a model take? We calculate the number of weights of a neural network
def count_weights(layers):
    """layers: the number of neurons in each layer, for example [64, 64, 10]."""
    total = 0
    for n_in, n_out in zip(layers, layers[1:]):
        total += n_in * n_out + n_out        # weights + one threshold for each neuron
    return total


networks = {
    "our network for digits (64-64-10)": [64, 64, 10],
    "a network for 96 × 96 images (9216-128-10)": [96 * 96, 128, 10],
    "a bigger network (9216-512-256-10)": [96 * 96, 512, 256, 10],
}
chips = {"Arduino Uno": 2, "micro:bit V2": 128, "ESP32": 520}     # working memory (RAM) in KB

for name, layers in networks.items():
    n = count_weights(layers)
    kb_float = n * 4 / 1024        # a decimal number (float) takes 4 bytes
    kb_int8 = n * 1 / 1024         # a “shrunk” model: one byte per weight
    print(f"{name}: {n} weights = {kb_float:.0f} KB (float) or {kb_int8:.0f} KB (int8)")
    for chip, ram in chips.items():
        print(f"    {chip:12} ({ram:3} KB RAM): {'fits' if kb_int8 < ram * 0.7 else 'does NOT fit'}")

Program output:

our network for digits (64-64-10): 4810 weights = 19 KB (float) or 5 KB (int8)
    Arduino Uno  (  2 KB RAM): does NOT fit
    micro:bit V2 (128 KB RAM): fits
    ESP32        (520 KB RAM): fits
a network for 96 × 96 images (9216-128-10): 1181066 weights = 4614 KB (float) or 1153 KB (int8)
    Arduino Uno  (  2 KB RAM): does NOT fit
    micro:bit V2 (128 KB RAM): does NOT fit
    ESP32        (520 KB RAM): does NOT fit
a bigger network (9216-512-256-10): 4853002 weights = 18957 KB (float) or 4739 KB (int8)
    Arduino Uno  (  2 KB RAM): does NOT fit
    micro:bit V2 (128 KB RAM): does NOT fit
    ESP32        (520 KB RAM): does NOT fit

• Our network for digits from module 3 has 4810 weights – it fits on both the micro:bit V2 and the ESP32.
• An ordinary network for 96 × 96 pixel images is too big. That is why special convolutional networks are used for images; they “slide” the same small group of weights over the whole image – they have far fewer weights, so they can fit on an ESP32-S3.
• This calculation is simplified: the weights can be kept in flash memory, and RAM is also needed for intermediate results. But it shows the key thing – the number of weights decides whether a model fits.

How a model is “shrunk”

Tools that are used

• TensorFlow Lite for Microcontrollers (Google; since 2024 the library has been part of a project called LiteRT) – runs small neural networks on microcontrollers, including the ESP32. The network is trained in Python, then converted into an array of bytes that is inserted into the program.
• Edge Impulse – a website where you record data from a board or phone, train a model and get a ready library for the Arduino IDE. It requires an account (for students – with a teacher and parents).
• ESP-DL and ESP-SR (Espressif) – libraries for recognizing faces, objects and voice commands on ESP32-S3 chips.
• micro:bit CreateAI – gesture recognition on the micro:bit, without programming.

For school, the easiest path is to start with trees and k-NN (as in this module), and only then move on to tools for neural networks.

Examples of TinyML devices

Common misconceptions

• “TinyML is the same as a chatbot on a board” – large language models have billions of weights and don't fit on a microcontroller; TinyML means small models for one task.
• “A model on a chip is trained on the chip” – it is almost always trained on a computer, and the finished model is moved to the chip.

Practice tasks

Try it yourself first, and only then open the solution.

1. How many weights does a network with layers of 64, 16 and 10 have? How many kilobytes is that as float, and how many as int8?
2. For a “smart doorbell” that recognizes the faces of the people living in the house – would the model run in the cloud or on the device? Give your reasons.
3. Why are models almost always trained on a computer and not on a microcontroller?

What's next

You have finished the module AI and the Internet of Things. Next is the module AI and society, which starts with the lesson Is artificial intelligence conscious? In it we talk about big questions: does artificial intelligence think, how can it be fair and safe, and what is its future.

Check your knowledge