To get started with either the Mini PCIe or M.2 Accelerator, all you need to do is connect the card to your system, and then install our PCIe driver, Edge TPU runtime, and the TensorFlow Lite runtime. This page walks you through the setup and shows you how to run an example model.
The setup and operation is the same for both form-factors.
To get started, you need a Linux computer with the following specs:
- x86-64 or ARM64 system architecture
- An available Mini PCIe or M.2 key slot
- Debian 6.0 Linux distribution or higher, or any derivative thereof (such as Ubuntu 10.0+)
- Python 3.5 or higher
1. Install the PCIe driver
Make sure the host system where you'll connect the module is shut down.
Connect the Coral PCIe/M.2 module according to your host system recommendations.
Boot the system and make sure it has an internet connection. Then install our PCIe driver with these commands:
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo apt update
sudo apt install gasket-dkms
If the user account you'll be using does not have root permissions, you might need to add the following udev rule, and then verify that the "apex" group exists and that your user is added to it:
sudo sh -c "echo 'SUBSYSTEM==\"apex\", MODE=\"0660\", GROUP=\"apex\"' >> /etc/udev/rules.d/65-apex.rules"
sudo groupadd apex
sudo adduser $USER apex
Reboot the system.
Verify that the accelerator module is detected:
lspci -x | grep 089a
You should see something like this:
03:00.0 System peripheral: Device 1ac1:089a
System peripheralname might be different, because those are host-system specific, but as long as you see a device listed with
089athen you're okay to proceed.
Verify that the PCIe driver is loaded:
You should simply see the name repeated back:
2. Install the Edge TPU runtime
The Edge TPU runtime is required to communicate with the Edge TPU. You can install it on your host computer from a command line as follows.
First add our Debian package repository to your system:
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - sudo apt-get update
Then install the Edge TPU runtime:
sudo apt-get install libedgetpu1-std
3. Install the TensorFlow Lite library
There are several ways you can install TensorFlow's APIs, but to get started with Python,
the easiest option is to install the
tflite_runtime package. This package provides the bare
minimum code required to run an inference with Python (primarily, the
Interpreter API), thus saving you a lot of
To install it, follow the TensorFlow Lite Python quickstart, and then return to this page after you run the
pip3 install command.
4. Run a model using the TensorFlow Lite API
Now you're ready to run an inference on the Edge TPU. Follow these steps to perform image classification with our example code and model:
Download the example code from GitHub:
mkdir coral && cd coral
git clone https://github.com/google-coral/tflite.git
Download the bird classifier model, labels file, and a bird photo:
Run the image classifier with the bird photo (shown in figure 1):
python3 classify_image.py \ --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \ --labels models/inat_bird_labels.txt \ --input images/parrot.jpg
You should see results like this:
INFO: Initialized TensorFlow Lite runtime. ----INFERENCE TIME---- Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory. 11.8ms 3.0ms 2.8ms 2.9ms 2.9ms -------RESULTS-------- Ara macao (Scarlet Macaw): 0.76562
Congrats! You just performed an inference on the Edge TPU using TensorFlow Lite.
To demonstrate varying inference speeds, the example repeats the same inference five times. It prints the time to perform each inference and the top classification (the label ID/name and the confidence score, from 0 to 1.0). Your inference speeds might differ based on your host system.
classify_image.py example above uses the TensorFlow Lite Python API. To learn more
about how it works, take a look at the
classify_image.py source code
and read about how to run inference with TensorFlow Lite.
As an alternative to using the TensorFlow Lite API (used above), you can use the Edge TPU
Python API, which provides high-level APIs to perform inference with
image classification and object detection models with just a few lines of code. For an example, try
our other version of
classify_image.py using the Edge TPU API.
You can also run inference using C++ and TensorFlow Lite.
To run some other types of neural networks, check out our example projects, including examples that perform real-time object detection, pose estimation, keyphrase detection, on-device transfer learning, and more.
If you want to create your own model, see these pages:
Or to create your own model that's compatible with the Edge TPU, read TensorFlow Models on the Edge TPU.
The following section describes how the power throttling works and how to customize the trip points.
Operating frequency and thermal settings
The Mini PCIe/M.2 Accelerator does not include a thermal solution to dissipate heat from the system. In order to sustain maximum performance from the Edge TPU, it's important that you design your system so the Edge TPU operates well below the maximum operating temperature specified in the datasheet. If the Edge TPU gets too hot, it slowly reduces the operating frequency and may reset to avoid permanent damage.
The PCIe driver includes a power throttling mechanism (also known as dynamic frequency scaling) and an emergency shutdown mechanism, based on temperature readings from the Edge TPU. By default, this system checks the Edge TPU temperature every 5 seconds and responds as follows:
- If the Edge TPU is below 85°C, continue at the "maximum" operating frequency.
- If the Edge TPU reaches 85°C, reduce the operating frequency 50% (from "maximum" to "normal").
- If the Edge TPU reaches 90°C, reduce the operating frequency another 50% (from "normal" to "low").
- If the Edge TPU reaches 95°C, reduce the operating frequency yet another 50% (from "low" to "lowest").
- If the Edge TPU reaches 100°C, reset the Edge TPU.
By reducing the operating frequency, the Edge TPU's inferencing speed becomes slower, but it also consumes less power and hopefully avoids reaching the hardware reset threshold. But other heat sources in your system can also be a factor.
As long as the chip does not reset and the Edge TPU temperature returns to lower levels, the system restores the operating frequency in the reverse manner—ultimately returning to the maximum operating frequency when the Edge TPU is below 85°C.
Read the Edge TPU temperature
To avoid throttling at all, you might want to periodically read the temperature of the Edge TPU and—if the temperature is nearing the threshold for throttling (85°C, by default)—activate a thermal solution (such as a fan) or load-balance your work across other Edge TPUs in the system.
To read the temperature of an Edge TPU, you can use the following sysfs node:
The three directories in the middle (
pci0000:00/0000:00:01.0/000:01:00.0/) are unique for each
Coral PCIe module so will differ from the names shown here.
Customize the trip points
If necessary, you can also adjust the parameters that define the Edge TPU's throttling and reset trip points, effectively disabling all throttling and even disabling the emergency reset—although we recommend that you always keep the emergency reset enabled.
The temperature for each trip point is defined with a file accessible as either a kernel module parameter or a sysfs node:
The kernel module parameters are located in this path:
/sys/module/apex/parameters/These parameters are persistent and applied at boot time, and useful if you have multiple modules for which you want to apply the same settings.
The sysfs nodes for each module are located in paths such as this:
/sys/devices/_pci0000:00/0000:00:01.0/000:01:00.0_/apex/apex_0/The three directories in the middle (
pci0000:00/0000:00:01.0/000:01:00.0/) are unique for each Coral PCIe module. These sysfs nodes are created by the PCIe driver at boot time and allow you to set different settings for different modules.
/sys/module/apex/parameters/) are applied at system boot time, whereas the individual sysfs nodes can be changed at runtime and take immediate effect. To persistently change the kernel module parameters so you don't need to reapply the configuration upon every boot, read how to specify kernel module parameters.
Whether you decide to use the kernel module parameters or the individual sysfs node parameters, the files that specify the trip points for frequency throttling and the hardware reset are named the same, as shown in table 1 and table 2.
||If the Edge TPU temperature reaches or exceeds this value, the system sets the operating frequency to "normal" (50% of maximum)||
||If the Edge TPU temperature reaches or exceeds this value, the system sets the operating frequency to "low" (25% of maximum)||
||If the Edge TPU temperature reaches or exceeds this value, the system sets the operating frequency to "lowest" (~13% of maximum)||
||The interval at which to read the temperature (setting to 0 disables polling)||
Whatever values you set for the
trip_point* parameters, they must evaluate as follows:
trip_point0_temp <= trip_point1_temp <= trip_point2_temp
If you set values that don't match this logic, then the driver reverts to use the default values
shown in table 1.
temp_poll_intervalbe a value of several seconds because the temperature doesn't change instantly (in response to changes in operational load or changes to the operating frequency). Yet, it also doesn't need to be much larger than the default value because the overhead of switching the operating frequency is negligible, so it isn't necessary to implement hysteresis around your trip points.
||If the Edge TPU temperature reaches or exceeds this value, the Edge TPU resets||
||Disable/enable emergency hardware reset||
||Boolean (1 is enabled, 0 is disabled)|
When the Edge TPU resets, it enters an idle state. Generally, you must then restart your system to resume work with the Edge TPU.
If you are running on ARM64 platform and receive error messages such as the following when you run an inference...
HIB Error. hib_error_status = 0000000000002200, hib_first_error_status = 0000000000000200
... You should be able to solve it if you modify your kernel command line arguments to include
For information about how to modify your kernel command line arguments, refer to your respective
platform documentation. For bootloaders based on U-Boot, you can usually modify the arguments either
by modifying the
bootargs U-Boot environment variable or by setting
variable as follows:
=> setenv othbootargs gasket.dma_bit_mask=32 => printenv othbootargs othbootargs=gasket.dma_bit_mask=32 => saveenv
If you make the above change and then receive errors such as,
DMA: Out of SW-IOMMU space, then
you need to increase the
swiotlb buffer size by adding another kernel command line argument:
Is this content helpful?