install_ubuntu

This document describes the installation process for WMD3/AmiRNA. WMD3 (Web Micro RNA Designer version 3) is a web application for artificial microRNA design based on the AmiRNA Perl module library. For more information please visit the website at http://wmd3.weigelworld.org. We focus on a single tier deployment although multi tier installations are possible. The recommended target platform is a Ubuntu 18.04 server system, but other Linux/UN*X flavors supporting Perl, MySQL, Apache and a C compiler will work as well.

This installation How To is split up into 7 sections: Section 2 describes the database setup. Section 3 deals with the installation and configuration of the AmiRNA Modules including the installation of its dependencies (some Perl modules, GenomeMapper, Blast, the ViennaRNA package and MySQL). Section 4 explains data preprocessing, section 5 the installation of the web application and finally section 6 discusses the designer daemon setup.

2 Database server setup

We support SQLite and MySQL as database backend. If you choose SQLite as backend you have to install the appropriate DBI drivers:

$ sudo apt update && sudo apt upgrade
$ sudo apt install libdbd-sqlite3 libdbd-sqlite3-perl

If you choose MySQL and you don't have a database server setup yet, install MySQL now:

$ sudo apt install mysql-server mysql-client
$ sudo mysql_secure_installation

$ sudo mysql

mysql> CREATE DATABASE amirnadb;
mysql> CREATE USER 'amirna'@'%' IDENTIFIED BY 'passwd';
mysql> GRANT ALL ON amirnadb.* TO 'amirna'@'localhost' identified by 'passwd';

3 AmiRNA Installation

3.1 Prerequisites

3.1.1 Third party Perl modules

The AmiRNA package relies on some third party Perl modules which are available as Ubuntu packages.

$ sudo apt install libxml-smart-perl \
libdbi-perl \
libdata-types-perl \
liberror-perl \
libxml-dumper-perl \
libdbd-mysql-perl \
liblog-log4perl-perl \
libgd-gd2-perl

$ cd /tmp
$ curl -O https://cpan.metacpan.org/authors/id/T/TM/TMHARISH/XML-Smart-1.78.tar.gz
$ tar xvzf XML-Smart-1.78.tar.gz
$ cd XML-Smart-1.78
$ perl Makefile.PL
$ make && sudo make install

3.1.2 Build environment

Some prerequisites like GenomeMapper and ViennaRNA needs to be compiled from source. Please check if you have a C/C++ compiler and the make utility installed.

$ sudo apt install gcc g++ make

3.1.3 GenomeMapper

Download and install GenomeMapper version 0.4.4 from the project site at http://1001genomes.org/downloads/genomemapper.html

$ cd /tmp
$ curl -O https://1001genomes.org/data/software/genomemapper/genomemapper_0.4.4/genomemapper-0.4.4.tar.gz
$ tar xvzf genomemapper-0.4.4.tar.gz
$ cd genomemapper-0.4.4 && make
$ sudo cp genomemapper gmindex /usr/local/bin

3.1.4 Vmatch

You can use Vmatch as a drop-in replacement for GenomeMapper. More information on Vmatch could be found at http://www.vmatch.de/

$ cd /tmp
$ curl -O http://www.vmatch.de/distributions/vmatch-2.3.0-Linux_x86_64-64bit.tar.gz
$ tar xvzf vmatch-2.3.0-Linux_x86_64-64bit.tar.gz
$ cd vmatch-2.3.0-Linux_x86_64-64bit
$ sudo cp vendian vmatchselect vseqinfo vstree2tex mkvtree mkdna6idx vmatch vmigrate.sh vseqselect vsubseqselect matchcluster chain2dim /usr/local/bin

3.1.5 ViennaRNA

AmiRNA needs RNAup and RNAcofold from the ViennaRNA package. Download and install version 2.4.10 or higher from http://www.tbi.univie.ac.at/RNA/.

$ cd /tmp
$ curl -O https://www.tbi.univie.ac.at/RNA/download/sourcecode/2_4_x/ViennaRNA-2.4.10.tar.gz
$ tar xvzf ViennaRNA-2.4.10.tar.gz
$ cd ViennaRNA-2.4.10/
$ ./configure
$ make && sudo make install

Note: Do not use the Ubuntu package as it is outdated. As well version 2.4.8 is known not to be compatible with AmiRNA.

3.1.6 NCBI Blast

The Blast tool is needed for the AmiRNA data preprocessing procedure and the WMD3 web application.

sudo apt install blast2

3.2 Installation

3.2.1 AmiRNA Modules

$ cd /tmp
$ curl -O http://wmd3.weigelworld.org/downloads/wmd3-3.2-rc3.tar.gz
$ tar xvzf wmd3-3.2-rc3.tar.gz
$ cd wmd3-3.2-rc3/AmiRNA/
$ perl Makefile.PL
$ make && sudo make install

3.2.2 AmiRNA Tools

$ sudo cp tools/Preprocess/* /usr/local/bin
$ sudo cp tools/amirna_initdb.pl /usr/local/bin

3.3 AmiRNA Configuration

$ sudo mkdir /etc/AmiRNA/
$ sudo cp etc/AmiRNA.xml /etc/AmiRNA/

$ sudo mkdir -p /var/opt/amirna/tmp
$ sudo chmod ugo+rwx /var/opt/amirna/tmp/

The configuration file is /etc/AmiRNA/AmiRNA.xml. Open it with your favorite text editor and adjust it for your needs.

$ sudo vim /etc/AmiRNA/AmiRNA.xml

Set the path where your transcript libraries will be stored. Please create this directory if it does not exist.

<index_path>/var/biodata/amirna/genomemapper</index_path>

Set the directory where temporary files will be created. The default should work in the most cases.

<tmp_path>/var/opt/amirna/tmp</tmp_path>

Note: Do not use tmp as Ubuntu 18.04 uses systemd's privatetmp which is not compatible with WMD3.

Configure the database. Use the same database name and user as in the database setup above. For MySQL please specify the database name, host and port in the <dsn> directive:

<database>
    <engine>MySQL</engine>
    <dsn>DBI:mysql:database=amirnadb;host=localhost;port=3306</dsn>
    <user>amirna</user>
    <password>passwd</password> 
</database>

For SQLite please specify the location of the database file (dbname) in the <dsn> section:

<database>
     <engine>SQLite</engine>
     <dsn>dbi:SQLite:dbname=/var/biodata/amirna/amirna.db</dsn>
     <user>amirna</user>
     <password>passwd</password>
</database>

Choose the mapping tool. Currently supported are genomemapper and vmatch. Please specify the name of the binary and the path in the mapping section. For the folding section specify the path of RNAup and RNAcofold. If you are unsure about the paths, try 'which genomemapper' and 'which RNAcofold'.

<external_programs>x
    <!-- Path and binary of user selectable mapping tool;
         supported: vmatch and genomemapper -->
    <mapping>
        <path>/usr/local/bin</path>
        <binary>genomemapper</binary>
    </mapping>

    <!-- Path to ViennaRNA tools RNAcofold and RNAup -->
    <folding>
        <path>/usr/local/bin</path>
    </folding>
</external_programs>

3.4 Database Setup

Initially create the queue and results relations. amirna_initdb.pl uses the database defined in /etc/AmiRNA/AmiRNA.xml.


$ amirna_initdb.pl

This initialization script should create three database tables. You can verify this with:


$ mysql -u amirna -D amirnadb -p -e "show tables;"

 +--------------------+
 | Tables_in_amirnadb |
 +--------------------+
 | queue              |
 | results            |
 | sequences          |
 +--------------------+

4. Data preprocessing

4.1 Generate mapping index

$ sudo mkdir -p /var/biodata/amirna/genomemapper
$ cd /tmp
$ curl -O http://wmd3.weigelworld.org/downloads/Araport11_genes.201606.cdna.fasta.gz
$ gunzip Araport11_genes.201606.cdna.fasta.gz
$ sudo generate_mapping_index.pl -d "Araport11 genes 201606 cdna" -f Araport11_genes.201606.cdna.fasta

4.2 Generate Blast index

$ sudo mkdir -p /var/biodata/amirna/blast
$ sudo cp /var/biodata/amirna/genomemapper/Araport11_genes.201606.cdna.fasta /var/biodata/amirna/blast/
$ cd /var/biodata/amirna/blast/
$ sudo formatdb -p F -i Araport11_genes.201606.cdna.fasta

5 WMD3 Setup

5.1 Prerequisites

5.1.1 Apache Webserver

$ sudo apt install apache2

$ sudo mkdir -p /var/www/html/wmd3
$ sudo mkdir -p /usr/lib/cgi-bin/wmd3/

Add the following lines to your vhost configuration (e.g. /etc/apache2/sites-available/000-default.conf)

DocumentRoot /var/www/html/wmd3
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/wmd3/
<Location /cgi-bin>
    Options +ExecCGI
</Location>

$ sudo a2enmod cgid
$ sudo systemctl restart apache2

5.1.2 Imagemagick

$ sudo apt install imagemagick

We need to configure imagemagick to enable PostScript processing. Please edit /etc/ImageMagick-6/policy.xmland change the line

<policy domain="coder" rights="none" pattern="PS" />

<policy domain="coder" rights="read" pattern="PS" />

5.1.3 Perl Modules

$ sudo apt install libcgi-application-perl \
libcgi-application-basic-plugin-bundle-perl \
libhtml-template-perl \
libspreadsheet-writeexcel-perl

We need to install CGI-Application-Plugin-View-HTML-Template and CGI-Application-Plugin-Stash from source as there are no Ubuntu packages:

$ cd /tmp
$ curl -O curl -O https://cpan.metacpan.org/authors/id/M/MF/MFULLER/CGI-Application-Plugin-View-HTML-Template-0.02.tar.gz
$ tar xzvf CGI-Application-Plugin-View-HTML-Template-0.02.tar.gz
$ cd CGI-Application-Plugin-View-HTML-Template-0.02
$ perl Makefile.PL
$ make && sudo make install

$ cd /tmp
$ curl -O https://cpan.metacpan.org/authors/id/K/KA/KAZEBURO/CGI-Application-Plugin-Stash-0.01.tar.gz
$ tar xvzf CGI-Application-Plugin-Stash-0.01.tar.gz
$ cd CGI-Application-Plugin-Stash-0.01/
$ perl Makefile.PL
$ make && sudo make install

5.2 WMD3 Installation

$ cd /tmp
$ curl -O http://wmd3.weigelworld.org/downloads/wmd3-3.2-rc3.tar.gz
$ tar xvfz wmd3-3.2-rc3.tar.gz
$ cd wmd3-3.2-rc3

$ sudo mkdir -p /usr/lib/cgi-bin/wmd3
$ sudo cp -r WMD3/lib/WMD3 WMD3/webapp.cgi WMD3/wmd3.xml /usr/lib/cgi-bin/wmd3

$ sudo mkdir -p /var/www/html/wmd3/
$ sudo cp -r WMD3/layout_files WMD3/index.html /var/www/html/wmd3/

$ sudo mkdir -p /var/www/tmp
$ sudo chown www-data:www-data /var/www/tmp

$ sudo vim /usr/lib/cgi-bin/wmd3/wmd3.xml

and adjust the URL prefix of the static files (htdocs), the URL of the web application (cgi), the path of the static web root (webroot) and the administrator email adress. In the section 'amirna' please specify the location of your AmiRNA.xml configuration file. If you followed the instructions above the default values will work for you.

<web_app>
     <htdocs>http://localhost/wmd3</htdocs>
     <cgi>http://localhost/cgi-bin/wmd3/webapp.cgi</cgi>
     <webroot>/var/www/wmd3</webroot>
     <admin_mail>wmdhelp@localhost</admin_mail>
     <standart_project>stdwmd</standart_project>
</web_app>

<amirna>
     <configfile>/etc/AmiRNA/AmiRNA.xml</configfile>
     <default_genome>Araport11_genes.201606.cdna.fasta</default_genome>
</amirna>

<blast>
     <db_location>/var/biodata/amirna/blast</db_location>
</blast>

6 designerd

6.1 Prerequisites

You need a configured AmiRNA installation (see section 3), a Database (section 2) and preprocessed data (section 4).

If you intend to setup a computation node hosting only designerd, you can configure designerd to use a remote database. Index files must be accessible for this host over the filesystem either as a local copy or shared via a network filesystem. For more information see Appendix A.

6.2 Installation

$ cd /tmp
$ curl -O http://wmd3.weigelworld.org/downloads/wmd3-3.2-rc3.tar.gz
$ tar xvzf wmd3-3.2-rc3.tar.gz
$ cd wmd3-3.2-rc3/
$ sudo cp AmiRNA/tools/Designerd/designerd.pl /usr/sbin/
$ sudo cp AmiRNA/tools/Designerd/designerd.xml /etc/AmiRNA/
$ sudo touch /var/log/designerd_log
$ sudo cp AmiRNA/tools/Designerd/designerd.service /lib/systemd/system/designerd.service
$ sudo systemctl daemon-reload

6.3 Configuration

$ sudo vim /etc/AmiRNA/designerd.xml

Please specify the location of your AmiRNA configuration file (amirna_config), your mailhost, email address and the base URL of your WMD3 installation:

<config>
    <amirna_config>/etc/AmiRNA/AmiRNA.xml</amirna_config>
    <max_childs>2</max_childs>
    <no_jobs_sleep_time>10</no_jobs_sleep_time>
    <pid_file>/var/run/designerd.pid</pid_file>
    <log_file>/var/log/designerd_log</log_file>
    <mailhost>mailgw.localhost</mailhost>
    <from_email>wmd3@localhost</from_email>
    <wmd3_webapp_url>http://localhost/cgi-bin/webapp.cgi</wmd3_webapp_url>
</config>

6.4 Start designerd

Designerd is implemented as a UN*X deamon running in the background. Use init.d scripts to start and stop the deamon:

sudo systemctl start designerd.service

$ tail -f /var/log/designerd_log

sudo systemctl enable designerd.service

Appendix

A. Setting up a designerd computation node

It's possible to run only designerd on a host without a local database or web application installed on this host. One or more designerds share the queue, results and sequences database. This allows a distributed setup for parallel designer job processing.

# Prerequisites
$ sudo apt-get install libxml-smart-perl libdbi-perl libdata-types-perl
liberror-perl libxml-dumper-perl libdbd-mysql-perl libgd2-xpm-dev liblog-
log4perl-perl gcc g++ make

# GenomeMapper
$ curl -O https://1001genomes.org/data/software/genomemapper/genomemapper_0.4.4/genomemapper-0.4.4.tar.gz
$ tar xvzf genomemapper-0.4.4.tar.gz
$ cd genomemapper-0.4.4 && make
$ sudo cp genomemapper gmindex /usr/local/bin

# ViennaRNA
curl -O https://www.tbi.univie.ac.at/RNA/download/sourcecode/2_4_x/ViennaRNA-2.4.10.tar.gz
tar xvzf ViennaRNA-2.4.10.tar.gz
cd ViennaRNA-2.4.10/
sudo apt install g++
./configure
make && sudo make install

$ cd /tmp; wget http://wmd3.weigelworld.org/downloads/wmd3-3.2-rc3.tar.gz
$ tar xvfz wmd3-3.2-rc3.tar.gz
$ cd wmd3-3.2-rc3/AmiRNA/
$ perl Makefile.PL
$ make
$ sudo make install

sudo cp AmiRNA/tools/Designerd/designerd.pl /usr/sbin/
sudo cp AmiRNA/tools/Designerd/designerd.xml /etc/AmiRNA/
sudo touch /var/log/designerd_log
sudo cp etc/AmiRNA.xml /etc/AmiRNA/
sudo cp AmiRNA/tools/Designerd/designerd.service /lib/systemd/system/designerd.service
sudo systemctl daemon-reload
sudo systemctl enable designerd.service

Now configure AmiRNA. You need to specify a local path to the GenomeMapper / Vmatch index files and a remote database.

Next, run the data preprocessing script and generate the index files (See section “Generate mapping index”, you do not need to generate blast indices). As final step you have to configure and start designerd (see the appropriate section above).

B. Upgrading

# Stop services
$ sudo service designerd stop
$ sudo service apache2 stop

$ sudo vim /etc/apache2/sites-available/wmd3
# Disable CGI
  AliasMatch ^/cgi-bin(.*) /var/www/wmd3/down.html

$ sudo service restart

# WMD3 web application
$ sudo rm -rf /usr/lib/cgi-bin/wmd3/ /var/www/wmd3/
# designerd
$ sudo rm -rf /usr/sbin/designerd.pl /lib/systemd/system/designerd.service

# AmiRNA Perl modules
$ sudo rm -rf /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/auto/AmiRNA/ /usr/local/share/perl/5.26.1/AmiRNA/

# Reset queue
$ mysql -u wmd_user -p wmd3 -e "UPDATE queue SET started=0 WHERE
started=1; SELECT id, started FROM queue;"

Installation Guide for AmiRNA/WMD3 Release 3.2

Table of Contents

1 Introduction