======Setting up An Application Server Centos 6.5 64bit ====== Create a folder to download and store the necessary software\\ mkdir -p /soft Turn off the firewall from starting up on system restart\\ chkconfig iptabes off chkconfig ip6tables off Turn off selinux from interfering with installs, ports\\ sed -i 's/enforcing/disabled/g' /etc/selinux/config Update default system libraries\\ yum update init 6 (reboot System) We will need to install the following as we will be compiling some packages\\ yum install nano lynx wget vim at yasm zip unzip nasm bzip2 libyaml libyaml-devel -y yum groupinstall "Development Tools" -y yum groupinstall "Additional Development" -y yum install xorg-x11-server-Xvfb -y ====== Install EPEL and additional repositories on CentOS ====== yum install epel-release -y ====== Install sendmail ====== Remove postfix \\ yum remove postfix yum install sendmail chkconfig sendmail on ====== Install Openssl ====== cd /soft && wget http://www.openssl.org/source/openssl-1.0.1j.tar.gz cd /soft && tar -xvf openssl-1.0.1j.tar.gz && cd openssl-1.0.1j && ./config --prefix=/usr --openssldir=/usr/local/openssl shared -fPIC && make && make install ln -s /usr/bin/openssl yum install openssl-devel create symlink for new version of openssl \\ ln -s /usr/bin/openssl ====== remove localtime ====== rm /etc/localtime ====== set Los Angeles time zone ====== ln -s /usr/share/zoneinfo/America/Los_Angeles /etc/localtime apache service dependency \\ yum install pcre pcre-devel -y ====== Add user ====== adduser expprodl login as expprodl ssh-keygen -t rsa mkdir -p ~/.ssh/ cd ~/.ssh/ cat id_rsa.pub >> authorized_keys chmod 700 ~/.ssh/ chmod 640 ~/.ssh/authorized_keys ===== Grant sudo power to expprodl ===== * vi /etc/sudoers (Add below mentioned lines) ## Drivers #Cmnd_Alias DRIVERS = /sbin/modprobe Cmnd_Alias APACHE = /opt/apache2/bin/apachectl, /bin/vi /opt/apache2/conf/httpd.conf Cmnd_Alias MYSQL = /sbin/service mysql stop, /sbin/service mysql start, /sbin/service mysql status, /sbin/service mysql restart Cmnd_Alias IPTABLES = /sbin/service iptables stop, /sbin/service iptables start, /sbin/iptables expprodl ALL= NOPASSWD: APACHE, MYSQL, IPTABLES ==== Install Apache ==== cd /tmp wget --user=expbbbu --password= ftp://qacollab01.exphosted.com/softrepo/app/installed-software/httpd-2.2.21.tar tar xvf httpd-2.2.21.tar ./configure --prefix=/opt/apache2 --enable-mods-shared=most --enable-ssl --enable-usertrack --enable-unique-id --with-included-apr --with-pcre=/usr/bin/pcre-config make && make install echo 'export APXS2=/opt/apache2/bin/apxs' >> /etc/profile.d/crossbow.sh logout === Verify === /opt/apache2/bin/apachectl start lynx localhost Expected output: You should see a default page text. ==== Install Apache modules ==== cd /tmp wget --user=expbbbu --password= ftp://qacollab01.exphosted.com/softrepo/app/installed-software/apache-upload-progress-module.tar tar xvf apache-upload-progress-module.tar cd apache-upload-progress-module sudo /opt/apache2/bin/apxs -c -i -a mod_upload_progress.c == Verify == ls /opt/apache2/modules/ | grep mod_upload_progress.so Expected output: mod_upload_progress.so Additionally, repeat the verification steps from Apache Installation section above. === For Production server follow below link ruby version ruby-enterprise-1.8.7-2011.03 === Note : For prodcution server we are using ruby & dependecies are old versions https://wiki.exphosted.com/doku.php/setup_an_app_server === Setup Ruby-1.9.3-p550 === wget http://pyyaml.org/download/libyaml/yaml-0.1.5.tar.gz && tar xvzf yaml-0.1.5.tar.gz && ./configure --prefix=/usr/local && make && make install yum remove libxml2-devel-2.7.6-17.el6_6.1.x86_64 yum remove libxml2-python-2.7.6-17.el6_6.1.x86_64 wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/home:/pessimism:/toolreplacements/CentOS_CentOS-6/x86_64/libxml2-2.9.0-3.3.x86_64.rpm rpm -Uvh libxml2-2.9.0-3.3.x86_64.rpm wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/home:/pessimism:/toolreplacements/CentOS_CentOS-6/x86_64/libxml2-devel-2.9.0-3.3.x86_64.rpm yum localinstall libxml2-devel-2.9.0-3.3.x86_64.rpm wget http://cache.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p550.tar.gz tar xvf ruby-1.9.3-p550.tar.gz ./configure --prefix=/opt/ruby-1.9.3 make make install echo 'pathmunge /opt/ruby-1.9.3/bin' >> /etc/profile.d/crossbow.sh ln -s /opt/ruby-1.9.3/bin/ruby /usr/bin/ ln -s /opt/ruby-enterprise-1.8.7-2011.03/bin/god /usr/bin/ ln -s /opt/ruby-1.9.3/bin/gem /usr/bin/ Verify ruby -v ==== Update rubygems ==== gem update --system 1.8.29 ===== Install passenger ===== gem install passenger -v=4.0.44 Make changes to the apache config file. (Config file remains same as the one used for ruby 1.8.7 REE's passenger 3 but the following lines need to be modified. #LoadModule passenger_module /opt/ruby-enterprise-1.8.7-2011.03/lib/ruby/gems/1.8/gems/passenger-3.0.9/ext/apache2/mod_passenger.so LoadModule passenger_module /opt/ruby-1.9.3/lib/ruby/gems/1.9.1/gems/passenger-4.0.44/buildout/apache2/mod_passenger.so LoadModule upload_progress_module modules/mod_upload_progress.so PassengerRoot /opt/ruby-1.9.3/lib/ruby/gems/1.9.1/gems/passenger-4.0.44 PassengerDefaultRuby /opt/ruby-1.9.3/bin/ruby #RailsFrameworkSpawnerIdleTime 0 PassengerMaxPreloaderIdleTime 0 #RailsAppSpawnerIdleTime 0 PassengerMaxPreloaderIdleTime 0 ### End Passenger tuning## #RailsAutoDetect off ====== Update the gem versions of the components ====== If the following gems are already installed, please uinstall them using “gem uninstall ” command and reinstall. gem install bundler -v=1.6.2 gem install rack -v "1.5.2" gem install god -v=0.13.3 gem install capistrano -v=2.15.4 gem install capistrano-ext -v=1.2.1 ====== Integrate Apache and Passenger ====== /opt/ruby-1.9.3/bin/passenger-install-apache2-module You will see a GUI. Keep on pressing next with a space bar and it will install the required module. \\ Verify \\ This will be automatically verified in the next section. ====== Setup Apache Conf files ====== cd /opt/apache2/conf wget --user=expbbbu --password= ftp://qacollab01.exphosted.com/softrepo/app/conf/httpd.conf_live wget --user=expbbbu --password= ftp://qacollab01.exphosted.com/softrepo/app/conf/httpd.conf_maintenance cp httpd.conf_live httpd.conf Verify \\ IP addresses in the conf file - By Default, Apache listens on All IP addresses. Additionally, repeat the verification steps from Apache Installation section above. Make sure the Rails Env variable is correctly set. ====== Update ruby path as needed ====== * For God - Change ruby path in /deplo/systasks/god.sh and /etc/init.d/god * For Capistrano - Change ruby path in /home/expprodl/crossbow/config/deploy.rb and /home/expprodl/crossbow/runcap.sh ==Install needed gems before hand== === Install Nokogiri === yum install libxml2 libxml2-devel (2.9.0-4) bundle config build.nokogiri --use-system-libraries gem install nokogiri -- --use-system-libraries -v='1.6.3.1' ==== Install RedCloth ==== bundle config build.RedCloth --with-cflags=-w gem install RedCloth -v='4.2.9' ===== Database config file change ===== Change /deploy/crossbow/shared/config/database.yml “adapter” to mysql2 in all the needed stanzas. ====== God configuration changes ====== Change /deploy/crossbow/shared/config/god/generic_monitoring.god to update the bundle paths to /opt/ruby-1.9.3/bin/bundle from /usr/local/bin/bundle ====== Install Capistrano ====== Please update the capistrano config files as needed. Refer the link below for more information. [[https://wiki.exphosted.com/doku.php/qastageandproductiondeployment#deployment_steps]] ====== Run db:seed ====== If you are using a fresh database for the current app setup, run the following task in the app directory bundle exec rake db:seed RAILS_ENV= ====== Install JRE ====== yum install jre-7u45-linux-x64.rpm yasm yasm-devel verify \\ java -version Expected output \\ java version "1.7.0_45" Java(TM) SE Runtime Environment (build 1.7.0_45-b18) Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode) ==== Install FFMPEG ==== wget --user=expbbbu --password= ftp://qacollab01.exphosted.com/softrepo/app/installed-software/ffmpeg-1.2.4-withmods.tar.gz tar xvzf faac-1.28.tar.gz cd faac-1.28 ./bootstrap cd ../faac-1.28/common/mp4v2/ vi mpeg4ip.h ## remove this line (new declaration ‘char* strcasestr(const char*, const char*) cd ../../ ./configure --enable-shared make make install cd lame-3.99.5 ./configure --enable-shared make make install ldconfig cd libogg-1.3.1 ./configure autoreconf --force --install make && make install cd speex-1.2rc1 ./configure make make install yum install libogg-devel yasm cd libtheora-1.1.1 ./configure --enable-shared make make install make check cd libvorbis-1.3.3 ./configure make make install cd libvpx/ ./configure make make install make && make install cd x264/ ./configure --enable-shared --disable-asm make make fprofiled make install ./configure --datadir=/usr/share/ffmpeg --extra-cflags=-I/tmp/ffmpeg-1.2.4/include --extra-ldflags=-L/tmp/ffmpeg-1.2.4/lib --bindir=/usr/local/bin --extra-libs=-ldl --enable-gpl --enable-nonfree --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libtheora --enable-libspeex --enable-libfaac make make install create symlink for dependency ffmpeg \\ ln -s /usr/local/lib/libx264.so.140 /usr/lib64/libx264.so.140 ln -s /usr/local/lib/libtheoraenc.so.1 /usr/lib64/libtheoraenc.so.1 ln -s /usr/local/lib/libtheoradec.so.1 /usr/lib64/libtheoradec.so.1 ln -s /usr/local/lib/libspeex.so.1 /usr/lib64/libspeex.so.1 ln -s /usr/local/lib/libmp3lame.so.0 /usr/lib64/libmp3lame.so.0 ln -s /usr/local/lib/libmp3lame.so.0 /usr/lib64/libmp3lame.so.0 ln -s /usr/local/lib/libfaac.so.0 /usr/lib64/libfaac.so.0 ln -s /usr/local/lib/libfaac.so.0 /usr/lib64/libfaac.so.0 FFMPEG will be installed at /usr/local/bin. \\ Additionally, the ruby ffmpeg wrapper we use streamio-ffmpeg needs ffmpeg at /usr/bin. \\ ln -nfs /usr/local/bin/ffmpeg /usr/bin/ffmpeg If FFMPEG shows this error "ffmpeg: error while loading shared libraries: *.so*: cannot open shared object file: No such file or directory" Run this command; echo "/usr/local/lib" >> /etc/ld.so.conf ldconfig Verify \\ Expected output: \\ ffmpeg version 1.2.4 Copyright (c) 2000-2013 the FFmpeg developers built on Jan 7 2014 01:23:25 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-54) configuration: --datadir=/usr/share/ffmpeg --extra-cflags=-I/tmp/ffmpeg-1.2.4/include --extra-ldflags=-L/tmp/ffmpeg-1.2.4/lib --bindir=/usr/local/bin --extra-libs=-ldl --enable-gpl --enable-nonfree --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libtheora --enable-libspeex --enable-libfaac libavutil 52. 18.100 / 52. 18.100 libavcodec 54. 92.100 / 54. 92.100 libavformat 54. 63.104 / 54. 63.104 libavdevice 54. 3.103 / 54. 3.103 libavfilter 3. 42.103 / 3. 42.103 libswscale 2. 2.100 / 2. 2.100 libswresample 0. 17.102 / 0. 17.102 libpostproc 52. 2.100 / 52. 2.100 Hyper fast Audio and Video encoder usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}... Use -h to get full help or, even better, run 'man ffmpeg' ===== Install Apache_OpenOffice_4.0.1 ===== tar xvzf Apache_OpenOffice_4.0.1_Linux_x86-64_install-rpm_en-US.tar.gz cd en-US/RPMS yum localinstall *.rpm --nogpgcheck cd /opt/openoffice4/ /opt/openoffice4/program/soffice -headless -accept="socket,host=127.0.0.1,port=8100;urp;" -nofirststartwizard& Verify \\ netstat -anp | grep 8100 Expected output \\ tcp 0 0 127.0.0.1:8100 0.0.0.0:* LISTEN 15465/soffice.bin ==== PDFTK & pdf2json ==== wget ftp://195.220.108.108/linux/centos/5.11/os/x86_64/CentOS/libgcj-4.1.2-55.el5.x86_64.rpm rpm -Uvh libgcj-4.1.2-55.el5.x86_64.rpm wget http://pkgs.repoforge.org/pdftk/pdftk-1.44-2.el6.rf.x86_64.rpm yum localinstall pdftk-1.44-2.el6.rf.x86_64.rpm Yum will also install dependency packages: \\ (1/6): java-1.4.2-gcj-compat-1.4.2.0-40jpp.115.x86_64.rpm (2/6): jpackage-utils-1.7.3-1jpp.3.el5.noarch.rpm (3/6): gjdoc-0.7.7-12.el5.x86_64.rpm (4/6): antlr-2.7.6-4jpp.2.x86_64.rpm (5/6): pdftk-1.44-2.el5.rf.x86_64.rpm (6/6): libgcj-4.1.2-54.el5.x86_64.rpm ## Maintain Java Runtime Environment version 1.7.0_45 ln -fs /usr/java/jre1.7.0_45/bin/java /usr/bin/java ln -fs /usr/java/jre1.7.0_45/bin/javaws /usr/bin/javaws Verify \\ java -version pdftk --version Expected output: \\ java version "1.7.0_45" Java(TM) SE Runtime Environment (build 1.7.0_45-b18) Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode) pdftk 1.44 a Handy Tool for Manipulating PDF Documents Copyright (C) 2003-10, Sid Steward - Please Visit: www.pdftk.com This is free software; see the source code for copying conditions. There is NO warranty, not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ===== Install pdf2json ===== tar xvf pdf2json-0.61.tar.gz cd pdf2json* && ./configure && make && make install yum install giflib-devel freetype-devel Verify \\ pdf2json Expected output: \\ pdf2json version 0.61 http://flexpaper.devaldi.com/pdf2json/, based on Xpdf version 3.02 ====== Install swftools ====== PDF2SWF Dependencies yum install giflib-devel freetype-devel wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/home:/Kris_Shannon:/RHEL_or_CentOS_Extras/CentOS_6/x86_64/swftools-0.9.2-3.1.el6.sis.x86_64.rpm rpm -ivh swftools-0.9.2-3.1.el6.sis.x86_64.rpm Verify pdf2swf -V Expected output: pdf2swf - part of swftools 0.9.2 ====== Install Mupdf ====== wget --user=expbbbu --password= ftp://qacollab01.exphosted.com/softrepo/app/installed-software/mupdf-1.2-source.zip unzip mupdf-1.2-source.zip cd mupdf-1.2-source make && make install Verify mupdf Expected output: usage: mupdf [options] file.pdf [page] -b - set anti-aliasing quality in bits (0=off, 8=best) -p - password -r - resolution ====== Install Microsoft Core Fonts ====== cd /tmp wget http://mirror.centos.org/centos/6/os/x86_64/Packages/ImageMagick-6.5.4.7-7.el6_5.x86_64.rpm wget ftp://mirror.switch.ch/pool/4/mirror/scientificlinux/6rolling/x86_64/os/Packages/ImageMagick-devel-6.5.4.7-7.el6_5.x86_64.rpm yum install ImageMagick-6.5.4.7-7.el6_5.x86_64.rpm yum install ImageMagick-devel-6.5.4.7-7.el6_5.x86_64.rpm wget --user=expbbbu --password= ftp://qacollab01.exphosted.com/softrepo/app/installed-software/msttcorefonts-2.5-1.noarch.rpm rpm -ivh msttcorefonts-2.5-1.noarch.rpm cd /usr/share/fonts/default/ ln -s ../msttcorefonts TrueType ==verify== This package only installs font files. You will not be able to install rmagic gem if this was unsuccesfull. === Install svn === yum install subversion -y wget http://mirror.centos.org/centos/6/os/x86_64/Packages/ImageMagick-6.5.4.7-7.el6_5.x86_64.rpm wget ftp://mirror.switch.ch/pool/4/mirror/scientificlinux/6rolling/x86_64/os/Packages/ImageMagick-devel-6.5.4.7-7.el6_5.x86_64.rpm rpm -ivh ImageMagick-6.5.4.7-7.el6_5.x86_64.rpm ImageMagick-devel-6.5.4.7-7.el6_5.x86_64.rpm ##Note Openssl version e and j cannot coexist if you have installed using rpm then; rpm -e openssl-1.0.1j-1.x86_64 yum install pcre-devel git mutt gcc gcc-c++ libxml2 libxml2-devel libxslt-devel libxslt mysql-devel make rpm -i openssl-1.0.1j-1.x86_64 --force # Verify openssl version # Expected OpenSSL 1.0.1j 15 Oct 2014 ==== Install rake ==== yum install pcre-devel imagemagick imagemagick-devel git mutt gcc gcc-c++ libxml2 libxml2-devel libxslt-devel libxslt mysql-devel make gem uninstall rake gem install rake -v 0.8.7 ===== Install God init.d ===== sudo mkdir /var/run/god sudo chown -R expprodl:expprodl /var/run/god ====== Install INIT file ====== cd /etc/init.d/ wget --user=expbbbu --password= ftp://qacollab01.exphosted.com/softrepo/app/conf/init/god ====== Download Precompiled binaries and scripts ====== Application Setup download from 10.229.94.13:/home/expprodl/crossbow.zip download from 10.229.94.13:/home/expprodl/apps.zip download from 10.229.94.13:/home/expprodl/bin.zip download from 10.229.94.13:/home/expprodl/deploy.zip cd /home/expprodl #unless this is a build server rm crossbow.zip Install the /home/expprodl/app folder cd /home/expprodl unzip app.zip rm app.zip Install the /home/expprodl/bin folder cd /home/expprodl unzip bin.zip rm bin.zip Install crossbow application root skeleton unzip deploy.zip chown -hR expprodl.expprodl deploy rm deploy.zip ==Detailed Log== 0.2 - added development libraries required for compiling native gem dependencies - gcc gcc-c++ libxml2-devel mysql-devel libxslt-devel make 0.1 - created. === Install Memcached === cd /tmp wget --user=expbbbu --password= ftp://qacollab01.exphosted.com/softrepo/app/installed-software/memcached-1.4.5-1.el5.x86_64.rpm rpm -ivvh memcached-1.4.5-1.el5.x86_64.rpm Modify the configuration so memcached allows network clients to connect: edit /etc/sysconfig/memcached OPTIONS="-l 0.0.0.0" Memcached is also monitored by God. == INIT Scripts == An Init script is automatically installed with the RPM version. You need to manually install an INIT script if building from source. Instructions here in the link above. Add memcached to list of services to be started at boot: chkconfig memcached on === Install Chat Role Node === We use Juggernaut which is a node application. At the time of this writing, it is deprecated. cd /tmp wget --user=expbbbu --password= ftp://qacollab01.exphosted.com/softrepo/app/installed-software/node-v0.2.4.tar.gz tar zxvf node-v0.2.4.tar.gz cd node-v0.2.4 ./configure make && make install ==== Install Binaries ==== The production user expprodl 's home directory is already shipped [[https://wiki.exphosted.com/doku.php/setup_an_app_server#Application Setup]] with the required Juggernaut and Redis software. #Ensure you are expprodl cd ~/bin chat_sss.sh start #Set permissions chown -R expprodl:expprodl /deploy/ chown -R expprodl:expprodl /home/expprodl/ chown -R expprodl:expprodl /opt/solr/ chown -R expprodl:expprodl /opt/tomcat6/ chown expprodl:expprodl /etc/init.d/god chown -R expprodl:expprodl /var/run/god chown -R expprodl:expprodl /soft/ chown -R expprodl:expprodl /opt/openoffice4/ chown -R expprodl:expprodl /opt/ruby*/ ==INIT Scripts== echo "Start Juggernaut, Redis" >> /etc/rc.local echo "/home/expprodl/bin/chat_sss.sh start" > /etc/rc.local === Installing Solr/Tomcat On Search Server === wget https://archive.apache.org/dist/tomcat/tomcat-6/v6.0.32/bin/apache-tomcat-6.0.32.tar.gz tar -xzf apache-tomcat-6.0.32.tar.gz mv apache-tomcat-6.0.32 /opt/tomcat6 wget https://archive.apache.org/dist/lucene/solr/1.4.1/apache-solr-1.4.1.tgz tar -xzf apache-solr-1.4.1.tgz cd apache-solr-1.4.1 cp -r example/solr /opt/ cp dist/apache-solr-1.4.1.war /opt/solr/ cp dist/apache-solr-1.4.1.war /opt/tomcat6/webapps/solr.war ###Set solr data path in /opt/solr/conf/solrconfig.xml (set to the shared drive on search server, unsure if this is required) mkdir -p /nfs00/share00/solr/data ###/opt/tomcat6/bin/catalina.sh (set JAVA_OPT under the while loop - mostly line #108 below done) JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/opt/solr" export JAVA_OPTS ##/opt/tomcat6/conf/tomcat-users.xml to include user/passwd ADD lines below ##/opt/tomcat6/conf/server.xml, add URIEncoding=“UTF-8” (mostly line # 69) ##Copy schema.xml from /deploy/crossbow/current/solr/conf (src = App server) to /opt/solr/conf (destination = Tomcat/Search server) ##Change ownership of /opt/solr and /opt/tomcat6 to the app user used to run tomcat. Note: Run the ln command below (on app servers), else deploy:cold (initial deploy task) will fail to install mysql gem ln -s /usr/lib64/mysql/libmysqlclient.a /usr/lib64/libmysqlclient.a /usr/bin/mysqladmin -u root password newpasswd Run the below as mysql root to create required database, user and privileges, adjust values as required: create database cb_production DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; grant all on cb_production.* to produser@'localhost' identified by 'prodpswd'; grant all on cb_production.* to produser@'websrvr_IP_or_NAME' identified by 'prodpswd'; login as app user = expprodl cd /deploy/crossbow/current bundle install bundle exec rake db:create --trace RAILS_ENV=staging bundle exec rake db:migrate --trace RAILS_ENV=staging bundle exec rake db:seed --trace RAILS_ENV=staging == Production post deployment check list == [[https://wiki.exphosted.com/doku.php/postdeploymentchecklist]] === Before start runcap check folder permissions === apache Mysql services only start by root , Remaining service must start & stop god with user privilege