Build MXNet

The natural language projects presented on this website use MXNet and GluonNLP to predict word context for a Sicilian dictionary and to develop a machine translator.

And because I enjoy programming in Perl, the MXNet in Perl pages also introduce the AI::MXNet module by Sergey Kolychev, which he uses for computer vision projects.

MXNet provides a good set of installation instructions, but any good introduction starts with an installation, so I typed a few notes on how I built and installed MXNet on Debian Buster and how I built the API bindings for Python and Perl.

set variables and paths

After building the shared library, I wanted to store it in $HOME/.local/mxnet/lib, so my first step was to add the following variables and paths to my $HOME/.bashrc:

## local library path -- contains libmxnet.so and more

export LD_LIBRARY_PATH="$HOME/.local/mxnet/lib"


## python path

export PYTHONPATH="$HOME/.local/lib/python3.7/site-packages"


## perl path

export PERL5LIB="$HOME/.perl/lib/perl5"


## set PATH so it includes $HOME/.perl/bin

PATH="$HOME/.perl/bin:$PATH"


## set PATH so it includes $HOME/.local/bin

PATH="$HOME/.local/bin:$PATH"


## set PATH so it includes $HOME/.bin

PATH="$HOME/.bin:$PATH"

install the build dependencies

Next, I installed the build dependencies for the shared library:

# apt-get install build-essential ca-certificates cmake git liblapack-dev libmkldnn-dev libopenblas-dev libopencv-dev ninja-build python3-dev unzip wget

Python's pip3 installer:

# apt-get install python3-pip

and the dependencies for the Perl module:

# apt-get install libmouse-perl pdl cpanminus swig libgraphviz-perl

build and install the shared library

Having chosen those paths and variables and having installed the build dependencies, I downloaded the apache-mxnet-src-1.4.1-incubating.tar.gz tarball from GitHub and began to build.

First, I extracted the tarball and entered the directory:

$ tar -xzf apache-mxnet-src-1.4.1-incubating.tar.gz

$ cd apache-mxnet-src-1.4.1-incubating/

$ MXNET_HOME=${PWD}

Then, I built the shared library (saving a copy of the build log) using OpenCV for computer vision, OpenBLAS for linear algebra and Intel's Math Kernel Library for Deep Neural Networks (MKL-DNN):

$ make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas USE_MKLDNN=1

If you have a Nvidia graphics card, you might want to use the CUDA toolkit:

USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1

After building the shared libraries, I installed them by copying them to $HOME/.local/mxnet/lib

$ cp lib/libiomp5.so ${LD_LIBRARY_PATH}/.

$ cp lib/libmkldnn.so.0 ${LD_LIBRARY_PATH}/.

$ cp lib/libmklml_intel.so ${LD_LIBRARY_PATH}/.

$ cp lib/libmxnet.a ${LD_LIBRARY_PATH}/.

$ cp lib/libmxnet.so ${LD_LIBRARY_PATH}/.

install the Python bindings

Next, I installed the Python package:

$ cd ${MXNET_HOME}/python

$ pip3 install .

I also had to create a few symbolic links inside the installed package:

$ cd $HOME/.local/lib/python3.7/site-packages/mxnet

$ ln -s $HOME/.local/mxnet/lib/libmxnet.so libmxnet.so

$ ln -s $HOME/.local/mxnet/lib/libmklml_intel.so libmklml_intel.so

$ ln -s $HOME/.local/mxnet/lib/libiomp5.so libiomp5.so

validate the installation

After that, I validated the installation by passing the following commands to the python3 interpreter:

>>> import mxnet as mx

>>> a = mx.nd.ones((2, 3))

>>> b = a * 2 + 1

>>> b.asnumpy()

which returned:

array([[3., 3., 3.],

       [3., 3., 3.]], dtype=float32)

which shows that MXNet and the Python bindings were successfully installed.

To test MKL-DNN, I also performed the validation at the MKL-DNN ReadMe page.

install the Perl bindings

Next, I installed the Perl packages:

$ cpanm -q -L "${HOME}/.perl" Function::Parameters Hash::Ordered PDL::CCS


$ cd ${MXNET_HOME}/perl-package/AI-MXNetCAPI/

$ perl Makefile.PL INSTALL_BASE="${HOME}/.perl"

$ make install


$ cd ${MXNET_HOME}/perl-package/AI-NNVMCAPI/

$ perl Makefile.PL INSTALL_BASE="${HOME}/.perl"

$ make install


$ cd ${MXNET_HOME}/perl-package/AI-MXNet

$ perl Makefile.PL INSTALL_BASE="${HOME}/.perl"

$ make test TEST_VERBOSE=1

$ make install


$ cd ${MXNET_HOME}/perl-package/AI-MXNet-Gluon-Contrib

$ perl Makefile.PL INSTALL_BASE="${HOME}/.perl"

$ make install


$ cd ${MXNET_HOME}/perl-package/AI-MXNet-Gluon-ModelZoo

$ perl Makefile.PL INSTALL_BASE="${HOME}/.perl"

$ make test TEST_VERBOSE=1

$ make install

And I validated its installation by passing the following commands to the pdl2 interpreter:

pdl> use AI::MXNet qw(mx)

pdl> $a = mx->nd->ones([2, 3])

pdl> $b = $a * 2 + 1

pdl> print $b->aspdl

which returned:

[

 [3 3 3]

 [3 3 3]

]

which shows that MXNet and the Perl bindings were successfully installed.

Copyright © 2002-2019 Eryk Wdowiak