Mastering Object Detection: Training YOLO on Custom Objects | by Frank Shane Alvares | Mar, 2025

Earlier than coaching YOLO, we have to construction our dataset correctly. In case you adopted Half 3, you need to have a well-annotated dataset from Label Studio. Now, let’s guarantee it’s within the appropriate format:

• Convert Annotations to YOLO Format: YOLO requires annotations in a selected format: every picture will get a corresponding textual content file the place every line represents an object with its class ID and bounding field coordinates (normalized between 0 and 1).
• Organizing Information: Create three directories inside your dataset folder:

- photographs/prepare (for coaching photographs)

- photographs/val (for validation photographs)

- labels/prepare and labels/val (for corresponding annotation recordsdata)

• Cut up the Dataset: Usually, we use an 80–20 break up between coaching and validation information. You should utilize Python scripts to automate this.

Now, we have to modify some configuration recordsdata so YOLO is aware of about our customized objects:

• Create a Knowledge Configuration File: This .yaml file tells YOLO the place to seek out coaching and validation information. It ought to look one thing like this:

prepare: /path/to/dataset/photographs/prepare val: /path/to/dataset/photographs/val nc:  names: ["class1", "class2", "class3"

• Modify the Model Configuration: Depending on the YOLO version you’re using, update the yaml or .cfg file to specify the number of classes in the last layer.
• Set Up Hyperparameters: Adjust batch size, learning rate, and epochs in the training script. A batch size of 16–32 and a learning rate of 0.01 are good starting points.

With the configuration set, we can now start training YOLO on our custom dataset.

1. Open a terminal and navigate to the YOLO directory.
2. Run the training command (for YOLOv11, it might look something like this):

python train.py --data custom_dataset.yaml --cfg yolov11.yaml --weights yolov11.pt --epochs 100 --batch-size 16 --device 

Additional Details: Here’s a breakdown of the parameters in your YOLO training command:

python train.py --data custom_dataset.yaml --cfg yolov11.yaml --weights yolov11.pt --epochs 100 --batch-size 16 --device 0

Additional Details: Parameter Breakdown
python train.py
Runs the training script (train.py), which starts the model training process.
--data custom_dataset.yaml
Specifies the dataset configuration file. This YAML file contains paths to training/validation images, number of classes, and class names.
--cfg yolov11.yaml
Defines the model configuration file. This file contains the YOLO network architecture settings, including layers, anchors, and other hyperparameters.
--weights yolov11.pt
Specifies the initial model weights.
If using a pre-trained model (e.g., yolov11.pt), it helps with transfer learning.
If set to """ --weights '' """, the model will train from scratch.
--epochs 100
The number of training iterations over the entire dataset.
More epochs generally improve accuracy but increase training time.
--batch-size 16
The number of images processed simultaneously during training.
A higher batch size speeds up training but requires more GPU memory.
--device 0
Specifies which device to use for training:
0: Use GPU (if available).
cpu: Train using the CPU (much slower).
0,1,2,...: Multi-GPU training (if multiple GPUs are available).

3. Monitor the training process. You should see metrics like loss, mAP (mean Average Precision), and training speed.

4. If loss doesn’t decrease, consider adjusting the learning rate or augmenting the dataset.

Once training completes, we need to evaluate performance:

• Check Precision and Recall: Look at the confusion matrix to see how well the model distinguishes between classes.
• Analyze mAP Score: The higher the mean Average Precision, the better the model’s accuracy.
• Test on New Images: Run inference on unseen images to visually inspect results.

Improving model performance often requires some tweaking:

• Data Augmentation: Apply techniques like flipping, rotation, and brightness adjustments.
• Hyperparameter Tuning: Experiment with different batch sizes, learning rates, and optimizer functions.
• Transfer Learning: If starting from scratch doesn’t yield good results, use pre-trained YOLO weights for better performance.