Commercial drone applications require significant autonomy and intelligence which cannot be achieved using conventional Computer Vision algorithms. Deep Learning/Vision algorithms with Nvidia Tx1 seem to be very promising in helping drones to be used in more advanced and complex commercial applications.
Getting a deep learning application working perfectly on a desktop is nontrivial, and when that application has to run on a single board computer aka Companion Computer controlling a drone, the task becomes quite challenging.
FlytOS provides a framework to alleviate these challenges by helping in an easy integration of your deep learning application with drones. It provides you the right set of Drone APIs and Drone SDK to build advanced drone applications.
We will post a series of tutorials to get a simple Deep Learning application like this video
… up and working on a Nvidia TX1 board running FlytOS.
In this tutorial, we will integrate Caffe, a popular deep learning framework widely used for Deep Vision.
A little knowledge about ROS will be helpful in understanding the tutorial better.
Making a catkin_ws:
We will have to make a catkin workspace for the ROS package we are going to write. If you already have a catkin workspace set up on your TX1, skip this step. In our case our workspace name is flytos_tx1. Clone the following repository: https://github.com/flytbase/flytos_tx1 as a reference to this tutorial.
Change your directory to ~/flytos_tx1/src/ssd_caffe/ and clone the caffe repository here by typing following commands:
git clone https://github.com/weiliu89/caffe.git
git checkout ssd_caffe
Now we have to make and install Caffe. You can refer the following link http://www.jetsonhacks.com/2016/09/18/caffe-deep-learning-framework-64-bit-nvidia-jetson-tx1/ as it is a nice resource on building Caffe for the TX1. Or follow these instructions:
Install some dependencies by typing the following commands:
sudo apt-get install cmake libprotobuf-dev libleveldb-dev libsnappy-dev libhdf5-serial-dev protobuf-compiler libatlas-base-dev libgflags-dev libgoogle-glog-dev liblmdb-dev python-dev python-numpy
sudo apt-get install –no-install-recommends libboost-all-dev
Then copy the example
Makefile.config.example into a file name
cp Makefile.config.example Makefile.config
If you want to enable the use of CuDNN libraries for accelerated performance, uncomment the line
Makefile.config. Then enter the following command:
Next, add the following line at the end of
Makefile.config to solve some issues related to making Caffe on Ubuntu 16.04:
INCLUDE_DIRS += /usr/include/hdf5/serial/
You can finally make caffe by typing these 2 commands:
make -j4 all
Compiling nodes in our ROS package:
If you look at the CMake file our package ssd_caffe (
~/flytos_tx1/src/ssd_caffe/CMakeLists.txt), you can see the 2 lines help Catkin in finding the required Caffe libraries to be linked to your ROS nodes.
set(CAFFE_INCLUDEDIR caffe/include /usr/local/cuda/include)
set(CAFFE_LINK_LIBRARY caffe/lib /usr/local/cuda/lib64)
Change your directory back to the catkin workspace and enter the following command to compile your code:
Right now there is just an empty file in the source directory.
Get the Next Tutorial Here.
Thanks to Zubin and Pradeep for preparing this demo.