"If you are not using Google Colab, it is recommended to create a separate [Python virtual environment](https://docs.python.org/3/library/venv.html) before you run this project. This ensures that packages installed for the conversion process do not conflict with other packages you may already have set up.\n",
"\n",
"## ⚠️ Linux Only\n",
"This notebook can only by run on **Linux** because the `rknn-toolkit2` Python package only supports Linux builds.\n",
"If you don’t have access to a Linux system, consider using a cloud service like [Google Colab](https://colab.research.google.com).\n"
"This notebook requires the use of custom Python code. Please run the installation script below to import these external scripts if you do not have them already.\n",
"\n",
"**You may need to run the `Create ONNX/RKNN` cell multiple times when you restart your session. If you see a `create_onnx`\n",
"or `create_rknn` not found error, rerun the cell below and then retry.**\n",
"\n",
"**Do not modify the cells in this setup section unless you know what you’re doing or have a specific reason to.**"
"#### *Numpy Fix* - Important for Google Colab Users\n",
"\n",
"Google Colab comes with an incompatible version of Numpy installed. To fix this, please run the following cells below and **restart your session** when prompted."
"To convert to ONNX, simply run the `create_onnx` function, providing the path to your model weights and specifying the model version, as shown below."
],
"id": "332942d8582c9bae"
},
{
"metadata": {},
"cell_type": "code",
"outputs": [],
"execution_count": null,
"source": "create_onnx(model_path=\"weights.pt\", version=\"yolov8\") # Valid versions are yolov5, yolov8, and yolov11",
"id": "408440b32de224ed"
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"### Step 2: Download RKNN API\n",
"\n",
"You can either use `pip` below to automatically detect and install the correct RKNN API Python library for you, or install it manually.\n",
"\n",
"#### Automatic installation\n",
"\n",
"Please run `pip` below. If it does not work, refer to the instructions for manual installation. You may need to restart your session after running the command below.\n"
"Visit the [RKNN Toolkit 2](https://github.com/airockchip/rknn-toolkit2) Github repository, then click on rknn-toolkit2, followed by packages.\n",
"If you are running an x86_64 CPU (e.g., most Intel and AMD processors), select that option; otherwise, choose arm64 for ARM-based computers (e.g., M-series Macs or Snapdragon processors). If you're unsure which CPU you're using, check your system settings for processor architecture information.\n",
"\n",
"Once you've selected the correct CPU architecture, you'll see multiple packages. The file names will look something like:\n",
"The numbers after cp correspond to your Python version. For example, if you're using Python 3.10, look for a package with cp310 in the name. For Python 3.8, look for cp38; for Python 3.7, cp37, and so on.\n",
"\n",
"Once you've found the correct package, click the \"Raw\" button to download the .whl file. Then, run the following command in your terminal, replacing rknn_toolkit2.whl with the actual path to the file you downloaded:"
],
"id": "f44b71c5a820ab2"
},
{
"metadata": {},
"cell_type": "code",
"outputs": [],
"execution_count": null,
"source": "%pip install rknn_toolkit2.whl",
"id": "c9b9d3da532eb916"
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"### Step 3: Convert to RKNN\n",
"\n",
"Please review the notes about quantization before running the RKNN conversion."
],
"id": "fc74ebc99b596f76"
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"### Quantization\n",
"\n",
"When performing quantization, it is critical to provide representative images of the objects or scenes you are trying to detect. These images are used to calibrate the model’s internal activations and greatly influence the final performance.\n",
"\n",
"It is recommended to use 300–500 representative images that reflect the real-world input your model will encounter. As the old saying goes, it’s quality over quantity — having a diverse, relevant set matters more than simply having many images.\n",
"\n",
"Quantization will cause some loss in model accuracy. However, if your calibration images are chosen wisely, this accuracy drop should be minimal and acceptable. If the sampled images are too uniform or unrelated, your quantized model's performance may worsen significantly.\n",
"\n",
"The script will automatically sample representative images randomly from the provided dataset. While this usually works well, please verify that the dataset contains diverse and relevant examples of your target objects. As a reminder, the images used to quantize the model are stored in the text file specified by `--img_dataset_txt`.\n"
],
"id": "80cf63a7f16f9af"
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"### Optional: Download a dataset from Roboflow for quantization\n",
"\n",
"If you do not already have a dataset or set of images containing the objects you want to detect, follow the steps below to download one from Roboflow Universe.\n",
"\n",
"#### **Step 1: Search for a Dataset**\n",
"\n",
"Go to [Roboflow Universe](https://universe.roboflow.com) and use the search bar to locate a dataset relevant to what you want to detect.\n",
"**Note:** The dataset must include the classes or object types you intend to detect.\n",
"\n",
"#### **Step 2: Access the Dataset Tab**\n",
"\n",
"After selecting a suitable project, navigate to the **Dataset** tab. Click the **\"Download Dataset\"** button. A prompt will appear with several options, including:\n",
"\n",
"- Train a model with this dataset\n",
"- Train from a portion of this dataset\n",
"- Download dataset\n",
"\n",
"Select **Download dataset**.\n",
"\n",
"#### **Step 3: Choose Format and View Download Code**\n",
"\n",
"- Under **Image and Annotation Format**, choose the version of YOLO you are using:\n",
" - For **YOLOv5**, choose `YOLOv5 PyTorch`\n",
" - For **YOLOv8**, choose `YOLOv8`\n",
" - For **YOLOv11**, choose `YOLOv11`\n",
"- If multiple annotation formats are listed for your model, always select the one ending in **\"PyTorch\"**.\n",
"\n",
"Then, under **Download Options**, click **\"Show Download Code\"** and continue.\n",
"\n",
"In the resulting screen, you will see three tabs:\n",
"- **Jupyter**\n",
"- **Terminal**\n",
"- **Raw URL**\n",
"\n",
"Select the **Terminal** tab and copy the provided command.\n",
"\n",
"#### **Step 4: Paste and Run**\n",
"\n",
"Paste the copied command into the notebook cell below and run it. This will download and extract the dataset into your environment, making it ready for use in the quantization process.\n",
"\n",
"Make sure to prefix the command with \"`!`\" so it executes properly in this Jupyter Notebook environment."
"To get started, run the `create_rknn` script below, replacing the arguments with your own values. Refer to the table below for detailed information on each argument’s purpose and usage. The `model_path` argument should point to your exported ONNX model from Step 1, and `img_dir` must reference a valid directory containing either a dataset or a set of images to be used for quantization.\n",
"\n",
"#### Overview of the `create_rknn` function\n",
"\n",
"This script converts a YOLO ONNX model to RKNN format using a set of calibration images. It's designed to work with either:\n",
"\n",
"- A flat directory of images (e.g. `train/images`), **or**\n",
"- A dataset directory containing a `data.yaml` file that defines `train`, `val`, and/or `test` folders.\n",
"| `img_dir` | `str` (required) | Path to your image directory. This can either be a folder of images **or** a dataset folder with a `data.yaml`. |\n",
"| `model_path` | `str` (required) | Path to your YOLO ONNX model, created in Step 1. |\n",
"| `num_imgs` | `int` (default: `300`) | Number of images to use for quantization calibration. |\n",
"| `disable_quantize` | `bool` (default: `False`) | Set to `True` to skip quantization entirely. Not recommended for performance, and should not be used for deployment on PhotonVision, which requires quantization. |\n",
"| `rknn_output` | `str` (default: `out.rknn`) | File path where the final RKNN model should be saved. |\n",
"| `img_dataset_txt` | `str` (default: `imgs.txt`) | File path to store the list of images used during quantization. |\n",
"| `verbose` | `bool` (default: `False`) | Enable detailed logging from the RKNN API during conversion. |\n",
"\n",
"\n",
"##### *Notes*\n",
"\n",
"1. This script is designed for use with [PhotonVision](https://photonvision.org), and by default sets the target platform for RKNN conversion to `RK3588`, a chipset commonly found in many variants of the Orange Pi 5 series (e.g., Orange Pi 5, 5 Pro, 5 Plus, 5 Max, etc.). You may modify the `DEFAULT_PLATFORM` value in the setup cell to match your specific hardware or deployment requirements if necessary.\n",
"\n",
"2. If you followed the Roboflow dataset download instructions from the previous section, the dataset will have been extracted to your **current working directory**. In that case, you can simply set `img_dir` to \"`.`\" to reference the current directory."
