Installation Guide for AmiRNA/WMD3 Release 3.2-rc3

November, 2018

Table of Contents

1 Introduction

2 Database server setup

3 AmiRNA Installation

3.1 Prerequisites

3.1.1 Third party Perl modules

3.1.2 Build environment

3.1.3 GenomeMapper

3.1.4 Vmatch

3.1.5 ViennaRNA

3.1.6 NCBI Blast

3.2 Installation

3.2.1 AmiRNA Modules

3.2.2 AmiRNA Tools

3.3 AmiRNA Configuration

3.4 Database Setup

4 Data preprocessing

4.1 Generate mapping index

4.2 Generate Blast index

5.1 Prerequisites

5.1.1 Apache Webserver

5.1.2 Imagemagick

5.1.3 Perl Modules

5.2 WMD3 Installation

6 designerd

6.1 Prerequisites

6.2 Installation

6.3 Configuration

6.4 Start designerd

Appendix

A. Setting up a designerd computation node

B. Upgrading

C. Uninstalling

1 Introduction

This document describes the installation process for WMD3/AmiRNA. WMD3 (Web Micro RNA Designer version 3) is a web application for artificial microRNA design based on the AmiRNA Perl module library. For more information please visit the website at http://wmd3.weigelworld.org. We focus on a single tier deployment although multi tier installations are possible. The recommended target platform is a Ubuntu 18.04 server system, but other Linux/UN*X flavors supporting Perl, MySQL, Apache and a C compiler will work as well.

This installation How To is split up into 7 sections: Section 2 describes the database setup. Section 3 deals with the installation and configuration of the AmiRNA Modules including the installation of its dependencies (some Perl modules, GenomeMapper, Blast, the ViennaRNA package and MySQL). Section 4 explains data preprocessing, section 5 the installation of the web application and finally section 6 discusses the designer daemon setup.

2 Database server setup

We support SQLite and MySQL as database backend. If you choose SQLite as backend you have to install the appropriate DBI drivers:

$ sudo apt update && sudo apt upgrade
$ sudo apt install libdbd-sqlite3 libdbd-sqlite3-perl

If you choose MySQL and you don't have a database server setup yet, install MySQL now:

$ sudo apt install mysql-server mysql-client
$ sudo mysql_secure_installation

Login as root and create a new user and grant some permissions on the new database:

$ sudo mysql
mysql> CREATE DATABASE amirnadb;
mysql> CREATE USER 'amirna'@'%' IDENTIFIED BY 'passwd';
mysql> GRANT ALL ON amirnadb.* TO 'amirna'@'localhost' identified by 'passwd';

3 AmiRNA Installation

3.1 Prerequisites

3.1.1 Third party Perl modules

The AmiRNA package relies on some third party Perl modules which are available as Ubuntu packages.

$ sudo apt install libxml-smart-perl \
libdbi-perl \
libdata-types-perl \
liberror-perl \
libxml-dumper-perl \
libdbd-mysql-perl \
liblog-log4perl-perl \
libgd-gd2-perl
$ cd /tmp
$ curl -O https://cpan.metacpan.org/authors/id/T/TM/TMHARISH/XML-Smart-1.78.tar.gz
$ tar xvzf XML-Smart-1.78.tar.gz
$ cd XML-Smart-1.78
$ perl Makefile.PL
$ make && sudo make install

3.1.2 Build environment

Some prerequisites like GenomeMapper and ViennaRNA needs to be compiled from source. Please check if you have a C/C++ compiler and the make utility installed.

$ sudo apt install gcc g++ make

3.1.3 GenomeMapper

Download and install GenomeMapper version 0.4.4 from the project site at http://1001genomes.org/downloads/genomemapper.html

$ cd /tmp
$ curl -O https://1001genomes.org/data/software/genomemapper/genomemapper_0.4.4/genomemapper-0.4.4.tar.gz
$ tar xvzf genomemapper-0.4.4.tar.gz
$ cd genomemapper-0.4.4 && make
$ sudo cp genomemapper gmindex /usr/local/bin

3.1.4 Vmatch

You can use Vmatch as a drop-in replacement for GenomeMapper. More information on Vmatch could be found at http://www.vmatch.de/

$ cd /tmp
$ curl -O http://www.vmatch.de/distributions/vmatch-2.3.0-Linux_x86_64-64bit.tar.gz
$ tar xvzf vmatch-2.3.0-Linux_x86_64-64bit.tar.gz
$ cd vmatch-2.3.0-Linux_x86_64-64bit
$ sudo cp vendian vmatchselect vseqinfo vstree2tex mkvtree mkdna6idx vmatch vmigrate.sh vseqselect vsubseqselect matchcluster chain2dim /usr/local/bin

3.1.5 ViennaRNA

AmiRNA needs RNAup and RNAcofold from the ViennaRNA package. Download and install version 2.4.10 or higher from http://www.tbi.univie.ac.at/RNA/.

$ cd /tmp
$ curl -O https://www.tbi.univie.ac.at/RNA/download/sourcecode/2_4_x/ViennaRNA-2.4.10.tar.gz
$ tar xvzf ViennaRNA-2.4.10.tar.gz
$ cd ViennaRNA-2.4.10/
$ ./configure
$ make && sudo make install

Note: Do not use the Ubuntu package as it is outdated. As well version 2.4.8 is known not to be compatible with AmiRNA.

3.1.6 NCBI Blast

The Blast tool is needed for the AmiRNA data preprocessing procedure and the WMD3 web application.

sudo apt install blast2

3.2 Installation

3.2.1 AmiRNA Modules

$ cd /tmp
$ curl -O http://wmd3.weigelworld.org/downloads/wmd3-3.2-rc3.tar.gz
$ tar xvzf wmd3-3.2-rc3.tar.gz
$ cd wmd3-3.2-rc3/AmiRNA/
$ perl Makefile.PL
$ make && sudo make install

3.2.2 AmiRNA Tools

$ sudo cp tools/Preprocess/* /usr/local/bin
$ sudo cp tools/amirna_initdb.pl /usr/local/bin

3.3 AmiRNA Configuration

Copy AmiRNA configuration to /etc/AmiRNA:

$ sudo mkdir /etc/AmiRNA/
$ sudo cp etc/AmiRNA.xml /etc/AmiRNA/

Create the tmp directory:

$ sudo mkdir -p /var/opt/amirna/tmp
$ sudo chmod ugo+rwx /var/opt/amirna/tmp/

The configuration file is /etc/AmiRNA/AmiRNA.xml. Open it with your favorite text editor and adjust it for your needs.

$ sudo vim /etc/AmiRNA/AmiRNA.xml

Set the path where your transcript libraries will be stored. Please create this directory if it does not exist.

<index_path>/var/biodata/amirna/genomemapper</index_path>

Set the directory where temporary files will be created. The default should work in the most cases.

<tmp_path>/var/opt/amirna/tmp</tmp_path>

Note: Do not use tmp as Ubuntu 18.04 uses systemd's privatetmp which is not compatible with WMD3.

Configure the database. Use the same database name and user as in the database setup above. For MySQL please specify the database name, host and port in the <dsn> directive:

<database>
    <engine>MySQL</engine>
    <dsn>DBI:mysql:database=amirnadb;host=localhost;port=3306</dsn>
    <user>amirna</user>
    <password>passwd</password> 
</database>

For SQLite please specify the location of the database file (dbname) in the <dsn> section:

<database>
     <engine>SQLite</engine>
     <dsn>dbi:SQLite:dbname=/var/biodata/amirna/amirna.db</dsn>
     <user>amirna</user>
     <password>passwd</password>
</database>

Choose the mapping tool. Currently supported are genomemapper and vmatch. Please specify the name of the binary and the path in the mapping section. For the folding section specify the path of RNAup and RNAcofold. If you are unsure about the paths, try 'which genomemapper' and 'which RNAcofold'.

<external_programs>x
    <!-- Path and binary of user selectable mapping tool;
         supported: vmatch and genomemapper -->
    <mapping>
        <path>/usr/local/bin</path>
        <binary>genomemapper</binary>
    </mapping>

    <!-- Path to ViennaRNA tools RNAcofold and RNAup -->
    <folding>
        <path>/usr/local/bin</path>
    </folding>
</external_programs>

3.4 Database Setup

Initially create the queue and results relations. amirna_initdb.pl uses the database defined in /etc/AmiRNA/AmiRNA.xml. $ amirna_initdb.pl

This initialization script should create three database tables. You can verify this with: $ mysql -u amirna -D amirnadb -p -e "show tables;"

The output should look like

 +--------------------+
 | Tables_in_amirnadb |
 +--------------------+
 | queue              |
 | results            |
 | sequences          |
 +--------------------+

4. Data preprocessing

4.1 Generate mapping index

Get some Fasta files and run generate_mapping_index.pl:

$ sudo mkdir -p /var/biodata/amirna/genomemapper
$ cd /tmp
$ curl -O http://wmd3.weigelworld.org/downloads/Araport11_genes.201606.cdna.fasta.gz
$ gunzip Araport11_genes.201606.cdna.fasta.gz
$ sudo generate_mapping_index.pl -d "Araport11 genes 201606 cdna" -f Araport11_genes.201606.cdna.fasta

4.2 Generate Blast index

$ sudo mkdir -p /var/biodata/amirna/blast
$ sudo cp /var/biodata/amirna/genomemapper/Araport11_genes.201606.cdna.fasta /var/biodata/amirna/blast/
$ cd /var/biodata/amirna/blast/
$ sudo formatdb -p F -i Araport11_genes.201606.cdna.fasta

5 WMD3 Setup

5.1 Prerequisites

5.1.1 Apache Webserver

Install the apache web server:

$ sudo apt install apache2

We need to create web root directories and configure Apache:

$ sudo mkdir -p /var/www/html/wmd3
$ sudo mkdir -p /usr/lib/cgi-bin/wmd3/

Add the following lines to your vhost configuration (e.g. /etc/apache2/sites-available/000-default.conf)

DocumentRoot /var/www/html/wmd3
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/wmd3/
<Location /cgi-bin>
    Options +ExecCGI
</Location>

Enable CGI support and restart Apache:

$ sudo a2enmod cgid
$ sudo systemctl restart apache2

5.1.2 Imagemagick

Install imagemagick:

$ sudo apt install imagemagick

We need to configure imagemagick to enable PostScript processing. Please edit /etc/ImageMagick-6/policy.xmland change the line

<policy domain="coder" rights="none" pattern="PS" />

to

<policy domain="coder" rights="read" pattern="PS" />

5.1.3 Perl Modules

Please install the following Perl Modules:

$ sudo apt install libcgi-application-perl \
libcgi-application-basic-plugin-bundle-perl \
libhtml-template-perl \
libspreadsheet-writeexcel-perl

We need to install CGI-Application-Plugin-View-HTML-Template and CGI-Application-Plugin-Stash from source as there are no Ubuntu packages:

$ cd /tmp
$ curl -O curl -O https://cpan.metacpan.org/authors/id/M/MF/MFULLER/CGI-Application-Plugin-View-HTML-Template-0.02.tar.gz
$ tar xzvf CGI-Application-Plugin-View-HTML-Template-0.02.tar.gz
$ cd CGI-Application-Plugin-View-HTML-Template-0.02
$ perl Makefile.PL
$ make && sudo make install

$ cd /tmp
$ curl -O https://cpan.metacpan.org/authors/id/K/KA/KAZEBURO/CGI-Application-Plugin-Stash-0.01.tar.gz
$ tar xvzf CGI-Application-Plugin-Stash-0.01.tar.gz
$ cd CGI-Application-Plugin-Stash-0.01/
$ perl Makefile.PL
$ make && sudo make install

5.2 WMD3 Installation

Download and untar the WMD3 archive:

$ cd /tmp
$ curl -O http://wmd3.weigelworld.org/downloads/wmd3-3.2-rc3.tar.gz
$ tar xvfz wmd3-3.2-rc3.tar.gz
$ cd wmd3-3.2-rc3

Install the application:

$ sudo mkdir -p /usr/lib/cgi-bin/wmd3
$ sudo cp -r WMD3/lib/WMD3 WMD3/webapp.cgi WMD3/wmd3.xml /usr/lib/cgi-bin/wmd3

$ sudo mkdir -p /var/www/html/wmd3/
$ sudo cp -r WMD3/layout_files WMD3/index.html /var/www/html/wmd3/

Create a temp directory:

$ sudo mkdir -p /var/www/tmp
$ sudo chown www-data:www-data /var/www/tmp

Configure the web application

$ sudo vim /usr/lib/cgi-bin/wmd3/wmd3.xml

and adjust the URL prefix of the static files (htdocs), the URL of the web application (cgi), the path of the static web root (webroot) and the administrator email adress. In the section 'amirna' please specify the location of your AmiRNA.xml configuration file. If you followed the instructions above the default values will work for you.

<web_app>
     <htdocs>http://localhost/wmd3</htdocs>
     <cgi>http://localhost/cgi-bin/wmd3/webapp.cgi</cgi>
     <webroot>/var/www/wmd3</webroot>
     <admin_mail>wmdhelp@localhost</admin_mail>
     <standart_project>stdwmd</standart_project>
</web_app>

<amirna>
     <configfile>/etc/AmiRNA/AmiRNA.xml</configfile>
     <default_genome>Araport11_genes.201606.cdna.fasta</default_genome>
</amirna>

<blast>
     <db_location>/var/biodata/amirna/blast</db_location>
</blast>

6 designerd

6.1 Prerequisites

You need a configured AmiRNA installation (see section 3), a Database (section 2) and preprocessed data (section 4).

If you intend to setup a computation node hosting only designerd, you can configure designerd to use a remote database. Index files must be accessible for this host over the filesystem either as a local copy or shared via a network filesystem. For more information see Appendix A.

6.2 Installation

$ cd /tmp
$ curl -O http://wmd3.weigelworld.org/downloads/wmd3-3.2-rc3.tar.gz
$ tar xvzf wmd3-3.2-rc3.tar.gz
$ cd wmd3-3.2-rc3/
$ sudo cp AmiRNA/tools/Designerd/designerd.pl /usr/sbin/
$ sudo cp AmiRNA/tools/Designerd/designerd.xml /etc/AmiRNA/
$ sudo touch /var/log/designerd_log
$ sudo cp AmiRNA/tools/Designerd/designerd.service /lib/systemd/system/designerd.service
$ sudo systemctl daemon-reload

6.3 Configuration

Open the configuration file for designerd:

$ sudo vim /etc/AmiRNA/designerd.xml

Please specify the location of your AmiRNA configuration file (amirna_config), your mailhost, email address and the base URL of your WMD3 installation:

<config>
    <amirna_config>/etc/AmiRNA/AmiRNA.xml</amirna_config>
    <max_childs>2</max_childs>
    <no_jobs_sleep_time>10</no_jobs_sleep_time>
    <pid_file>/var/run/designerd.pid</pid_file>
    <log_file>/var/log/designerd_log</log_file>
    <mailhost>mailgw.localhost</mailhost>
    <from_email>wmd3@localhost</from_email>
    <wmd3_webapp_url>http://localhost/cgi-bin/webapp.cgi</wmd3_webapp_url>
</config>

Please note that mailhost must reference a vaild SMTP server.

6.4 Start designerd

Designerd is implemented as a UN*X deamon running in the background. Use init.d scripts to start and stop the deamon:

sudo systemctl start designerd.service

You can monitor designerd's activity watching it's log file:

$ tail -f /var/log/designerd_log

If you would like designerd to be run at bootup you can enable it with

sudo systemctl enable designerd.service

Now you are ready to fire up your web browser and browse to http://localhost/cgi-bin/webapp.cgi.

Appendix

A. Setting up a designerd computation node

It's possible to run only designerd on a host without a local database or web application installed on this host. One or more designerds share the queue, results and sequences database. This allows a distributed setup for parallel designer job processing.

Install GenomeMapper, ViennaRNA and other prerequisites.

# Prerequisites
$ sudo apt-get install libxml-smart-perl libdbi-perl libdata-types-perl
liberror-perl libxml-dumper-perl libdbd-mysql-perl libgd2-xpm-dev liblog-
log4perl-perl gcc g++ make

# GenomeMapper
$ curl -O https://1001genomes.org/data/software/genomemapper/genomemapper_0.4.4/genomemapper-0.4.4.tar.gz
$ tar xvzf genomemapper-0.4.4.tar.gz
$ cd genomemapper-0.4.4 && make
$ sudo cp genomemapper gmindex /usr/local/bin

# ViennaRNA
curl -O https://www.tbi.univie.ac.at/RNA/download/sourcecode/2_4_x/ViennaRNA-2.4.10.tar.gz
tar xvzf ViennaRNA-2.4.10.tar.gz
cd ViennaRNA-2.4.10/
sudo apt install g++
./configure
make && sudo make install

Install AmiRNA:

$ cd /tmp; wget http://wmd3.weigelworld.org/downloads/wmd3-3.2-rc3.tar.gz
$ tar xvfz wmd3-3.2-rc3.tar.gz
$ cd wmd3-3.2-rc3/AmiRNA/
$ perl Makefile.PL
$ make
$ sudo make install

sudo cp AmiRNA/tools/Designerd/designerd.pl /usr/sbin/
sudo cp AmiRNA/tools/Designerd/designerd.xml /etc/AmiRNA/
sudo touch /var/log/designerd_log
sudo cp etc/AmiRNA.xml /etc/AmiRNA/
sudo cp AmiRNA/tools/Designerd/designerd.service /lib/systemd/system/designerd.service
sudo systemctl daemon-reload
sudo systemctl enable designerd.service

Now configure AmiRNA. You need to specify a local path to the GenomeMapper / Vmatch index files and a remote database.

Next, run the data preprocessing script and generate the index files (See section “Generate mapping index”, you do not need to generate blast indices). As final step you have to configure and start designerd (see the appropriate section above).

B. Upgrading

Clean old Installation:

# Stop services
$ sudo service designerd stop
$ sudo service apache2 stop

$ sudo vim /etc/apache2/sites-available/wmd3
# Disable CGI
  AliasMatch ^/cgi-bin(.*) /var/www/wmd3/down.html

$ sudo service restart

# WMD3 web application
$ sudo rm -rf /usr/lib/cgi-bin/wmd3/ /var/www/wmd3/
# designerd
$ sudo rm -rf /usr/sbin/designerd.pl /lib/systemd/system/designerd.service

# AmiRNA Perl modules
$ sudo rm -rf /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/auto/AmiRNA/ /usr/local/share/perl/5.26.1/AmiRNA/

# Reset queue
$ mysql -u wmd_user -p wmd3 -e "UPDATE queue SET started=0 WHERE
started=1; SELECT id, started FROM queue;"

Now you can proceed with the installation of AmiRNA and WMD3.