# object\_detection\_yolo\_setup.md

## By Peter Zhao

### Introduction

The FAQ section describes how to integrate YOLO with ros that allows the turtlebot to be able to detect object using its camera. [YOLO](https://arxiv.org/abs/1506.02640) (You only look once) is a widely used computer vision algorith that makes use of convolutional neural networks (CNN) to detect and label an object. Typically before a machine learning algorithm can be used, one needs to train the algorithm with a large data set. Fortunately, YOLO provides many pre-trained model that we can use. For example, yolov2-tiny weight can classify around 80 objects including person, car, bus, bir, cat, dog, and so on.

### darknet\_ros

In order to integrate YOLO with ROS easily, one can use this third party package known as [darknet\_ros](https://github.com/leggedrobotics/darknet_ros), which uses darknet (a neural network library written in C) to run YOLO and operates as a ros node. The node basically subscribes to the topics that has Image message type, and publishes bounding box information as [BoundingBoxes](https://github.com/leggedrobotics/darknet_ros/blob/master/darknet_ros_msgs/msg/BoundingBoxes.msg) message type to the /darknet\_ros/bounding\_boxes topic.

### How to use darknet\_ros

To use darknet\_ros, first clone the github repo, and place this directory somewhere inside your catkin\_workspace so that when you run catkin\_make, this can be built:

```bash
cd ~/catkin_ws/src/
git clone https://github.com/leggedrobotics/darknet_ros
```

Afterward, run catkin\_make to build the darknet\_ros package

```bash
cd ~/catkin_ws
catkin_make
```

You are now ready to run darknet\_ros!

### How to run darknet\_ros

darknet\_ros can be run aloneside with your other nodes. In your launch file include the following lines

```
<include file="$(find darknet_ros)/launch/darknet_ros.launch">
         <arg name="image" value="rapicam_node/image/raw"/>
</include>
```

Next you need to download the pretrained weight or put the weight you've trained yourself into the darknet\_ros/darknet\_ros/yolo\_network\_config/ folder. To download a pretrained weight, follow the insturction given [here](https://github.com/leggedrobotics/darknet_ros/blob/master/darknet_ros/yolo_network_config/weights/how_to_download_weights.txt). Essentially run the following commands

```
cd DIRECTORY_TO_DARKNET_ROS/darknet_ros/yolo_network_config/weights
wget http://pjreddie.com/media/files/yolov2-tiny.weights
```

Then run this launch file

```
roslaunch [your_package_name] [your_launch_file.launch]
```

You should be able to see the darknet\_ros node running with a window that displays the image with bounding boxes placed around objects. If you don't see any GUI window, try to check if the topic passed in in the "image" arg is publishing a valid image.

### Subscribing to darknet\_ros's topics

To receive information from the darknet\_ros topics, we need to subscribes to the topic it is publishing. Here is an example usage:

```python
from darknet_ros_msgs.msg import BoundingBoxes, BoundingBox

class ObjectRecognizer

    def __init__(self):
        self.boxes = []
        self.box_sub = box_sub = rospy.Subscriber(topics.BOUNDING_BOXES, BoundingBoxes, self.get_bounding_box_cb())

    def get_bounding_box_cb(self)
        
        def cb(msg: BoundingBoxes):
            self.boxes = msg.bounding_boxes
            for box in self.boxes:
                print("box_class: {}".format(box.Class))
                print("box_x_min: {}".format(box.xmin))
                print("box_x_max: {}".format(box.xmax))
                print("box_y_min: {}".format(box.ymin))
                print("box_y_max: {}".format(box.ymax))
                print()
```

The detailed description of the message types can be found in [darknet\_ros/darknet\_ros\_msg/msg](https://github.com/leggedrobotics/darknet_ros/tree/master/darknet_ros_msgs/msg) folder.

### Working with CompressedImage

You might have noticed that darknet\_ros expects to get raw image from the topic it subscribes to. Sometimes, we may want to use CompressedImage in order to reduce network latency so that the image can be published at a higher frame rate. This can be done by slightly modifying the source code of darknet\_ros.

I have forked the original darknet\_ros repository and made the modification myself. If you wish to use it, you can simply clone this [repo](https://github.com/zhaoy17/darknet_ros). Now you can modify the launch file to ensure that the darknet\_ros subscirbes to a topic that publishes CompressedImage message type:

```
<include file="$(find darknet_ros)/launch/darknet_ros.launch">
         <arg name="image" value="rapicam_node/image/compressed"/>
</include>
```

The changes I've made can be found in darknet\_ros/darknet\_ros/include/darknet\_rosYoloObjectDetector.hpp and darknet\_ros/darknet\_ros/src/YoloObjectDetector.cpp.

Essentially, you need to change the

```cpp
void YoloObjectDetector::cameraCallback(const sensor_msgs::CompressedImageConstPtr& msg) {
  ROS_DEBUG("[YoloObjectDetector] USB image received.");
```

method to accept msg of type sensor\_msgs::ImageConstPtr& instead of sensor\_msgs::CompressedImageConstPtr&. You also need to change the corresponding header file so that the method signature matches.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://campus-rover.gitbook.io/lab-notebook/fiiva/object_detection_yolo_setup.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.