cuDNN perf

nvvp in cuda 7.5 toolkit can not profile the cudnn.

but nvprof will work!


cudnn::detail::precomputed_convolve_sgemm
cudnn::detail::activation_fw_4d_kernel
kern_precompute_indices
add_tensor_kernel
cudnn::detail::softmax_fw_kernel
gemv2T_kernel_val
cudnn::detail::pooling_fw_4d_kernel

mnist using lenet (example in depth)

The input and output data is stored in lmdb format.
http://caffe.berkeleyvision.org/tutorial/data.html

when you run mnist using lenet, you can save the log file using the following command
./examples/mnist/train_lenet.sh > mylog  2>&1

It is observed that,  cudnn_conv_layer.cpp is used in convolution layer.

Solving LeNet starts from solver.cpp, and for every iteration it uses sgd_solver.cpp.

During training/testing, solver.cpp is used.
The Step() function is called, which use TestAll() function to check the performance.

in src/caffe/net.cpp, it uses ForwardFromTo(), to accumulate the loss using Forward() function.

net and layer use the shared_pointer.
like forward(), backward() is also switching between backward_cpu and backward_gpu.

forward_gpu() and backward_gpu() are all virtual functions.
Their specific implemantions are stored in .cu files, located src/caffe/layer/ folder.

in src/caffe/layer_factory.cpp, if using cudnn, the corresponding cudnn (conv, pooling, lrc ...) layer will be created.

When they process data, the blob (gpu) data is requested.
for convolution, the blobs for weight, bottom and top are needed.

src/caffe/synchedmem.cpp contains the gpu data transfer, sync or async.

For the forward algorithm in cudnn_conv_layer.cu,
=>leiming: FWD                                                                  
I0211 23:02:12.374897 18363 cudnn_conv_layer.cu:23] =>leiming: blobs[0] 20 1 5 5 (500)
I0211 23:02:12.374899 18363 cudnn_conv_layer.cu:24] =>leiming: bottom size :1   
I0211 23:02:12.374990 18363 cudnn_conv_layer.cu:30] =>leiming: bottom[0] size:100 1 28 28 (78400)
I0211 23:02:12.374997 18363 cudnn_conv_layer.cu:31] =>leiming: top[0] size:100 20 24 24 (1152000)
I0211 23:02:12.375000 18363 cudnn_conv_layer.cu:32] =>leiming: groups: 1        
I0211 23:02:12.375037 18363 cudnn_conv_layer.cu:48] =>leiming: using bias term: 1
I0211 23:02:12.375319 18363 cudnn_conv_layer.cu:51] =>leiming: blobs[1] 20 (20) 
I0211 23:02:12.375555 18363 cudnn_conv_layer.cu:22]

Which means that,
blobs, 20 number, 1 channel, width 5, height 5

MNIST + Caffe

(http://caffe.berkeleyvision.org/gathered/examples/mnist.html)

cd $CAFFE_ROOT
./data/mnist/get_mnist.sh
./examples/mnist/create_mnist.sh

Define MNIST Network

layers are defined in lenet_train_test.prototxt under $CAFFE_ROOT/examples/mnist/

protocol buffers – a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more.
It represents a serialized data structure, saved in .proto file. It is better than XML. (https://developers.google.com/protocol-buffers/docs/overview)

• are simpler
• are 3 to 10 times smaller
• are 20 to 100 times faster
• are less ambiguous
• generate data access classes that are easier to use programmatically

The protobuf for Caffe is stored in  $CAFFE_ROOT/src/caffe/proto/caffe.proto
A Blob is a wrapper over the actual data being processed and passed along by Caffe, and also under the hood provides synchronization capability between the CPU and the GPU. Mathematically, a blob is an N-dimensional array stored in a C-contiguous fashion.

lr_mults are the learning rate adjustments for the layer’s learnable parameters. In this case, we will set the weight learning rate to be the same as the learning rate given by the solver during runtime, and the bias learning rate to be twice as large as that - this usually leads to better convergence rates. (http://caffe.berkeleyvision.org/gathered/examples/mnist.html)

MNIST Solver

$CAFFE_ROOT/examples/mnist/lenet_solver.prototxt

Training and Testing the Model

cd $CAFFE_ROOT
./examples/mnist/train_lenet.sh

Data Flow In Details

[step 1]
~/caffe/tools/caffe.cpp
read the command line, calling ~/caffe/src/caffe/common.cpp
[step 2]
~/caffe/src/caffe/solver.cpp, initializing solver from parameters
[step 3]
solver creates a training net from net file: examples/mnist/lenet_train_test.prototxt
[step 4]
net.cpp, StateMeetsRule() function,
Caffe::root_solver(), Initializing net from parameters
[step 5]
In ~/caffe/include/caffe/layer_factory.hpp, CreateLayer() function
[step 6]
~/caffe/src/caffe/net.cpp
 LOG(INFO) << layer_param->name() << " -> " << blob_name;
[step 7]
read lmdb
[step 8]
set up mnist
    LOG_IF(INFO, Caffe::root_solver())
        << "Creating Layer " << layer_param.name();

[step 7]



time to create train and test networks :
21:32:45.919808 -  21:32:46.008837     Network initialization done.
less than 1 second

start optimization till the 10000 iterations
21:32:46.009042 -  21:33:24.866451     Optimization done.
39 seconds on GTX 760 using cuDNN

Tutorials and Learning Examples

[1] http://radar.oreilly.com/2014/07/how-to-build-and-run-your-first-deep-learning-network.html
[2] http://corpocrat.com/2014/11/03/how-to-setup-caffe-to-run-deep-neural-network/



It is recommended to learn from recognizing the digits, in order to understand the framework.
 

Examples_001_Classifying ImageNet: using the C++ API

A simple C++ code is proposed in examples/cpp_classification/classification.cpp.

The classification example will be built as examples/classification.bin in your build directory.

In my case, caffe/build/examples/cpp_classification/

go to script folder, run python script to download the model to bvlc_reference_caffenet

~/caffe/scripts$ ./download_model_binary.py ../models/bvlc_reference_caffenet/

(

You may need to install python-yaml,

sudo apt-get install python-yaml
sudo apt-get install libyaml-dev

)

got o ~/caffe/data/ilsvrc12

run ./get_ilsvrc_aux.sh , to obtain ImageNet labels file

classify the cat image (examples/iamges/cat.jpg) using this command:

./build/examples/cpp_classification/classification.bin \

models/bvlc_reference_caffenet/deploy.prototxt \

models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel \

data/ilsvrc12/imagenet_mean.binaryproto \

data/ilsvrc12/synset_words.txt \

examples/images/cat.jpg

install caffe on nvidia jetson tk1 (ubuntu 14.04)

[1] install cuda 

wget -c http://developer.download.nvidia.com/compute/cuda/6_5/rel/installers/cuda-repo-l4t-r21.2-6-5-prod_6.5-34_armhf.deb

sudo dpkg -i cuda-repo-l4t-r19.2_6.0-42_armhf.deb

sudo apt-get update

sudo apt-get install cuda-toolkit-6-5

sudo usermod -a -G video $USER

(add 32bit lib to bashrc)

echo "# Add CUDA bin & library paths:" >> ~/.bashrc

echo "export PATH=/usr/local/cuda/bin:$PATH" >> ~/.bashrc

echo "export LD_LIBRARY_PATH=/usr/local/cuda/lib:$LD_LIBRARY_PATH" >> ~/.bashrc

source ~/.bashrc

(to check the nvcc version)

$nvcc -V

(to install the sdk)

go to /usr/local/cuda/bin

./cuda-install-samples-6.5.sh /home/your_user_folder

Note: Many of the CUDA samples use OpenGL GLX and open graphical windows. If you are running these programs through an SSH remote terminal, you can remotely display the windows on your desktop by typing "export DISPLAY=:0" and then executing the program. (This will only work if you are using a Linux/Unix machine or you run an X server such as the free "Xming" for Windows). eg: 

export DISPLAY=:0

cd ~/NVIDIA_CUDA-6.5_Samples/2_Graphics/simpleGL

[2] install opencv

wget -c http://developer.download.nvidia.com/embedded/OpenCV/L4T_21.2/libopencv4tegra-repo_l4t-r21_2.4.10.1_armhf.deb

the downloaded file is libopencv4tegra-repo_l4t-r21_2.4.10.1_armhf.deb

$ sudo dpkg -i libopencv4tegra-repo_l4t-r21_2.4.10.1_armhf.deb

$ sudo apt-get update

$ sudo apt-get install libopencv4tegra libopencv4tegra-dev

If you bumped into the dependency issue

$sudo apt-get -f install

$ sudo apt-get install libopencv4tegra libopencv4tegra-dev

download cudnn from https://developer.nvidia.com/cuDNN

run the procedures in this linkhttps://gist.github.com/jetsonhacks/fa9f4ff89006607359ea
then you have your cudnn with it.
as default the tk1 uses 6.5, so I download the 6.5 compatible one, version 2.

the current caffe requires cudnn >3, i used v3 armv7

[3] get Caffe

go to https://gist.github.com/jetsonhacks
download the ./installCaffe.sh

#!/bin/sh
	# Install and compile Caffe on NVIDIA Jetson TK1 Development Kit
	sudo add-apt-repository universe
	sudo apt-get update
	sudo apt-get install libprotobuf-dev protobuf-compiler gfortran \
	libboost-dev cmake libleveldb-dev libsnappy-dev \
	libboost-thread-dev libboost-system-dev \
	libatlas-base-dev libhdf5-serial-dev libgflags-dev \
	libgoogle-glog-dev liblmdb-dev -y
	sudo usermod -a -G video $USER
	# Git clone Caffe
	sudo apt-get install -y git
	git clone https://github.com/BVLC/caffe.git
	cd caffe && git checkout dev
	cp Makefile.config.example Makefile.config
	make -j 4 all
	make -j 4 runtest
	build/tools/caffe time --model=models/bvlc_alexnet/deploy.prototxt --gpu=0

Noted that,

in order to use cudnn, you need to change makefile, turn on/add some options

http://elinux.org/Jetson/cuDNN

Since tk1 is cuda 3.2, I changed the arch in the Makefile.config to allow compiling only this architecture.

--------------------------------------------------------------------------------------------------------------------------

Reference:

*http://elinux.org/Jetson/Installing_CUDA

*https://developer.nvidia.com/linux-tegra-rel-21
*http://developer.download.nvidia.com/embedded/OpenCV/L4T_21.1/README.txt

* http://petewarden.com/2014/10/25/how-to-run-the-caffe-deep-learning-vision-library-on-nvidias-jetson-mobile-gpu-board/

Install caffe on ubuntu 14.04

Install cuda first, I am using 7.5

Install cuDNN

Go to https://developer.nvidia.com/cuDNN, download .tar.gz file and extract.
Here is the link for cuDNN v3 lib for linux.
wget -c http://developer.download.nvidia.com/assets/cuda/secure/cuDNN/v3/cudnn-7.0-linux-x64-v3.0-prod.tgz?autho=1445140377_f51da7032f9f7293550d875fc2b8c4ef&file=cudnn-7.0-linux-x64-v3.0-prod.tgz

v5 link:

http://developer.download.nvidia.com/compute/machine-learning/cudnn/secure/v5/rc/cudnn-7.5-linux-x64-v5.0-rc.tgz?autho=1460087472_a891d2a66796d775f8fc884a21ede27a&file=cudnn-7.5-linux-x64-v5.0-rc.tgz

Copy all the files, (except cudnn.h) to /usr/local/cuda-6.5/lib64

Copy the cudnn.h to /usr/local/cuda-6.5/include

Install cuda sdk
use the following example command (modify accordingly), the shell script is in bin dir of your cuda installation dir.
$cuda-install-samples-7.5.sh /home/xxxx/

Install opencv
http://rodrigoberriel.com/2014/10/installing-opencv-3-0-0-on-ubuntu-14-04/

you can donwload the opencv from the official website,
http://opencv.org/downloads.html

set cuda generation by auto

cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D WITH_V4L=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D CUDA_GENERATION=Auto ..

otherwise, you will see errors about gpu architecture 'compute_11'

refer to : https://help.ubuntu.com/community/OpenCV

install opencv 2.4.10

https://github.com/BVLC/caffe/wiki/Ubuntu-14.04-VirtualBox-VM

~/Downloads/opencv/opencv-3.0.0-alpha/modules/cudalegacy/src/cuda/NCVPixelOperations.hpp(51): error: a storage class is not allowed in an explicit specialization

if you are using opencv 3.0 alpha version, you may come across the errors below.

remove the static keyboard before the inline

do the same thing for

/home/leiming/Downloads/opencv/opencv-3.0.0-alpha/modules/cudastereo/src/cuda/stereocsbp.cu

line 62/66/74

/usr/include/opencv2/contrib/openfabmap.hpp:118:36: error: ‘vector’ does not name a type

put using namespace std before the included header files in openfabmap.hpp

Install caffe

Install all the dependencies and libraries needed for caffe in ubuntu

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler

sudo apt-get install libopenblas-dev
sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
sudo apt-get install --no-install-recommends libboost-all-dev

sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev
(installed libboost and libopenblas)

cp Makefile.config.example Makefile.config

In the Makefile.config, uncomment the "USE_CUDNN".
# cuDNN acceleration switch (uncomment to build with cuDNN).
# USE_CUDNN := 1

make -j $(nproc) all
make -j $(nproc) test 
make -j 8 runtest 
make pycaffe 
make distribute


/usr/bin/ld: cannot find -lcblas
/usr/bin/ld: cannot find -latlas

sudo apt-get install libatlas-base-dev
(http://caffe.berkeleyvision.org/install_apt.html)

make -j $(nproc) runtest


at last you should see your test passed!

Congratulations!

to test
build/tools/caffe time --model=models/bvlc_alexnet/deploy.prototxt --gpu=0


you can build pycaffe too.
(http://installing-caffe-the-right-way.wikidot.com/start)

To run caffe in eclipse, please refer to
http://tzutalin.blogspot.com/2015/05/caffe-on-ubuntu-eclipse-cc.html
http://tzutalin.blogspot.tw/2015/06/setup-caffe.html
You can build your caffe first, and add existing g++ project to eclipse, then configure the include, lib and preprocessor settings.


--------------------------------------------------------------------------------------------------------------------------
http://petewarden.com/2014/10/25/how-to-run-the-caffe-deep-learning-vision-library-on-nvidias-jetson-mobile-gpu-board/
http://corpocrat.com/2014/11/03/how-to-setup-caffe-to-run-deep-neural-network/