For new contributors of Percetion_CV team, please first create your own branch and make sure all your work is done within your branch. Do PR (pull request) only if your team leader asks you to do so.
For new team leaders of Perception_CV, the master branch should only contain stable code that has been confirmed working. Master branch will be the source we use for integration with other teams when the time is ready.
Full CV Repo here:
This repo is originally forked from but heavily modified for our own use. The purpose of this repo is to achieve custom object detection for Brandeis Autonomous Robotics Course. Changes were made based on our object of deploying CV on ROS.
To download our most recent best trained weights, please go to
Then unzip the file and copy coco and weights directory in this repo and replace everything.
Notes:
I've put a low of useful tools inside the ./utils directory, please feel free to use them whenever you need it.
./utils/vid_2_frm.py : The python script that extracts all frames out of a video, you can control the extracting rate by reading the comment and do small modification. This script will also tell you the fps of the source video which will be useful for later converting frames back to video.
./utils/frm_2_vid.py : The python script that is converting frames by its name into a video, you better know the original/target video's fps to get the optimal output.
Here are links to download our datasets (images and annotations) by certain class:
Doorplate Recognition:
custom_volen(provided by Haofan):
custom_doorplate(provided by Haofan):
Facial Recognition:
CV Subscriber & Publisher
All the CV subscriber and publisher are located at ./utils/ directory, they are:
./utils/image_subscriber.py : The python script that subscribe image from raspicam_node/image rostopic.
./utils/string_publisher.py : The python script that publishes a string on rostopic of /mutant/face_detection which is generated from detect.py, the format is explained below:
CV Publisher example: "['sibo', -4.34, 1.63]"
[
<"class name">,
<"angle of target to front in degree (negative -> left, positive -> right")>,
<"rough distance in meter">
]
Cheat Sheet For Raspberry Pi Camera
Detailed official user guide here:
Some useful commands:
raspivid -vf -hf -t 30000 -w 640 -h 480 -fps 25 -b 1200000 -p 0,0,640,480 -o pivideo.h264 recording 30 seconds video on 25 fps.
MP4Box -add pivideo.h264 pivideo.mp4 converting .h264 video to .mp4
scp donatello@129.64.243.61:~/pivideo.mp4 ~/Downloads/ downloading video from ssh to local machine
Pipeline of recording video on DONATELLO:
ssh donatello@129.64.243.61
If you want to see preview images, roslaunch turtlebot3_bringup turtlebot3_rpicamera.launch, then on remote computer, do rqt_image_view
when you recording video, shut down the rpicamera bringup
Cheat Sheet For USB Web-Camera
Get image_view
On slave, do lsusb and ls /dev |grep video to check if camera was recognized by system
Web Streaming
On slave, install web-video-server ROS node sudo apt install ros-kinetic-web-video-server
On slave, to make it right, create catkin workspace for our custom launch file mkdir -p ~/rosvid_ws/src
On slave, build package cd.. then catkin_make
On master, Make sure roscore is running
On slave, run created launch file roslaunch vidsrv vidsrv.launch
The repo contains inference and training code for YOLOv3 in PyTorch. The code works on Linux, MacOS and Windows. Training is done on the COCO dataset by default: . Credit to Joseph Redmon for YOLO: .
Python 3.7 or later with the following pip3 install -U -r requirements.txt packages:
Start Training: Run train.py to begin training after downloading COCO data with data/get_coco_dataset.sh.
Resume Training: Run train.py --resume resumes training from the latest checkpoint weights/latest.pt.
Each epoch trains on 117,263 images from the train and validate COCO sets, and tests on 5000 images from the COCO validate set. Default training settings produce loss plots below, with training speed of 0.6 s/batch on a 1080 Ti (18 epochs/day) or 0.45 s/batch on a 2080 Ti.
Here we see training results from coco_1img.data, coco_10img.data and coco_100img.data, 3 example files available in the data/ folder, which train and test on the first 1, 10 and 100 images of the coco2014 trainval dataset.
from utils import utils; utils.plot_results()
Image Augmentation
datasets.py applies random OpenCV-powered () augmentation to the input images in accordance with the following specifications. Augmentation is applied only during training, not during inference. Bounding boxes are automatically tracked and updated with the images. 416 x 416 examples pictured below.
Machine type: n1-standard-8 (8 vCPUs, 30 GB memory)
CPU platform: Intel Skylake
GPUs: K80 ($0.198/hr), P4 ($0.279/hr), T4 ($0.353/hr), P100 ($0.493/hr), V100 ($0.803/hr)
HDD: 100 GB SSD
Dataset: COCO train 2014
Run detect.py to apply trained weights to an image, such as zidane.jpg from the data/samples folder:
YOLOv3: python3 detect.py --cfg cfg/yolov3.cfg --weights weights/yolov3.weights
YOLOv3-tiny: python3 detect.py --cfg cfg/yolov3-tiny.cfg --weights weights/yolov3-tiny.weights
YOLOv3-SPP: python3 detect.py --cfg cfg/yolov3-spp.cfg --weights weights/yolov3-spp.weights
Run detect.py with webcam=True to show a live webcam feed.
Pretrained Weights
Darknet *.weights format:
Use test.py --weights weights/yolov3.weights to test the official YOLOv3 weights.
Use test.py --weights weights/latest.pt to test the latest training results.
Compare to darknet published results .
Issues should be raised directly in the repository. For additional questions or comments please contact your CV Team Leader or Sibo Zhu at siboz1995@gmail.com
