Look for at least few apache processes running:

/opt/apache2/bin/httpd -k start

Passenger processes are started in the following sequence (with start/restart of Apache web server)

Apache–>PassengerWatchdog–>PassengerHelperAgent–>Passenger spawn server–>“Passenger ApplicationSpawner: /deploy/crossbow/current”

PassengerLoggingAgent is also started by PassengerWatchdog

Ensure the below passenger processes are running:

				 
PassengerWatchdog
PassengerHelperAgent
Passenger spawn server                                                                                        
PassengerLoggingAgent

Below process is the only one restarted by the Application Deployment (on touch tmp/restart.txt)

Passenger ApplicationSpawner: /deploy/crossbow/current

Look for the below process, ensure its recently started:

/opt/ruby-enterprise-1.8.7-2011.03/bin/ruby /opt/ruby-enterprise-1.8.7-2011.03/bin/god -c /deploy/crossbow/current/config/production.god -P /var/run/god/god.pid

Look for below processes (started by god), ensure they are all recently started:

Memcached:

memcached -d -m 1000

Xvfb:

Xvfb :1 -screen 0 800x600x24

Starling:

ruby /deploy/crossbow/shared/bundle/ruby/1.8/bin/starling -d -p 15151 -P /deploy/crossbow/current/tmp/pids/starling.pid -q /deploy/crossbow/current/log/

Workling:

workling

Openoffice:

/usr/bin/soffice.bin -headless -accept=socket,host=127.0.0.1,port=8100;urp; -nofirststartwizard

Rufus:

ruby /deploy/crossbow/current/script/rufus

Chat processes are NOT restarted during the code upgrade, however, ensure they are running:

Redis:

./redis-server redis.conf

Node:

node server.js

Look for the below Tomcat_Solr (java process), ensure its recently started:

/usr/bin/java -Djava.util.logging.config.file=/opt/tomcat6/conf/logging.properties -Dsolr.solr.home=/opt/solr -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/tomcat6/endorsed -classpath /opt/tomcat6/bin/bootstrap.jar -Dcatalina.base=/opt/tomcat6 -Dcatalina.home=/opt/tomcat6 -Djava.io.tmpdir=/opt/tomcat6/temp org.apache.catalina.startup.Bootstrap start

mysql server process is NOT restarted during the code upgrade.
Only the DB backup is taken before “rake db:migrate”.
Look for backups under /data00/backups folder on master DB server (filename: DBNAME_YYYYMMDDHHMI.sql.gz)

Ensure new folder is created under /deploy/crossbow/releases and the /deploy/crossbow/current folder is linked to it.
By default, the following files under current/config folder are linked to the respective names in /deploy/crossbow/shared/config/

bigbluebutton.yml
database.yml
recorder.yml
<RAILS_ENV>.god
sunspot.yml
environments/<RAILS_ENV>.rb

While running the runcap.sh script to upgrade the code, if any errors are encountered by capistrano, a message as under is shown at the end of the script:

 === Errors encountered during the load, error file will be emailed.. ==== 

Error file production_cap_run_error.txt will be emailed, you can see the contents of this file in the same folder as runcap.sh script.

In order to keep persistent data between app nodes, we share some common folders between the nodes from a shared drive.

The following folders are shared among the nodes (mounted from a shared drive):

/deploy/crossbow/shared/assets
/deploy/crossbow/shared/system/uploaded_datas
/deploy/crossbow/shared/system/images

Links under /deploy/crossbow/current/public folder:

lrwxrwxrwx  1 expprodl expprodl     37 Feb 14 10:26 assets -> /deploy/crossbow/shared/assets/assets
lrwxrwxrwx  1 expprodl expprodl     48 Feb 14 10:26 homepage_features -> /deploy/crossbow/shared/assets/homepage_features
lrwxrwxrwx  1 expprodl expprodl     37 Feb 14 10:26 photos -> /deploy/crossbow/shared/assets/photos
lrwxrwxrwx  1 expprodl expprodl     30 Feb 14 10:26 system -> /deploy/crossbow/shared/system

After each deployment, verify that the links are valid, and contents under shared folder are the same (this ensures the shares are mounted and not stale mounts)

Besides, we also mount the /deploy/crossbow/shared/streams folder from the BBB server on each app node. After deploying BBB client and recorder code, verify on each node that this folder is in good mount state and not stale.

Create/login the test site - test.learnexa.com - and validate that all functions are working

Site Name: test (test.learnexa.com)
Admin user: test@learnexa.com
          password: Xr0ssb0w   (X is uppercase and 0 = zero)

The maintenance page is set for external (Internet) users during code deployment. After code is deployed, maintenance page will still remain active till we test the site/features/functionality using the Internal IP address of the web sites.

Login through the VPN using the username/password provided.

Edit your (local) hosts file to include the below IP addresses pointing to the respective sites.

10.166.152.249   learnexa.com
10.166.152.250   test.learnexa.com
10.166.152.247   www.methodyoga.com

Edit your (local) hosts file to include the below IP addresses pointing to the respective sites. <code> 10.166.152.17 learnexa.com 10.166.152.14 test.learnexa.com 10.166.152.19 www.methodyoga.com </code>

Begin testing (using domain names as usual), once done testing, comment out the above entries from your hosts file.

Maintenance page will be removed if things look correct and the site is ready to be released to the customers by selecting option 8) deploy:web:enable as Cap Task while running runcap.sh script. THIS FEATURE (of maintenance page) IS ONLY AVAILABLE ON PROD SETUP.