We currently have Newrelic and zabbix monitoring enabled in our NA3 environment;

For NewRelic please visit the following wiki;

https://wiki.exphosted.com/doku.php/appserver_monitoring

We are using Zabbix proxy in Active mode on Haproxy (10.229.94.14) on NA3 and all other servers pointing to proxy as in the below image.

Agent Configuration file = /etc/zabbix_agentd.conf

Proxy Configuration file = /etc/zabbix_proxy.conf

1. Login to server

create zabbix.repo in /etc/yum.repo.d/ with following content (Applicable for Current environment CentOS 6.x + Zabbix Server 2.x)

[zabbix]
name=Zabbix Official Repository - $basearch
baseurl=http://repo.zabbix.com/zabbix/2.2/rhel/6/$basearch/
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ZABBIX

[zabbix-non-supported]
name=Zabbix Official Repository non-supported - $basearch 
baseurl=http://repo.zabbix.com/non-supported/rhel/6/$basearch/
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-ZABBIX
gpgcheck=1

2. yum install zabbix22-agent.x86_64 (depends on the Zabbix Server version as well. Choose agent wisely)

3. Modify Zabbix agent configuration /etc/zabbix_agentd.conf

Hostname=<local_machine_name>
Server=<zabbix_proxy_ip>
ServerActive=<zabbix_proxy_ip>

4. Restart zabbix Agent /etc/init.d/zabbix-agentd restart

We are managing the system using TCP port listen state and custom scripts though Zabbix agent.

Here are the list of TCP/UDP ports which get Some important service in our application uses;

With PID and URL

God (On App servers only - PID)
Red5 (On Collab servers only and check following file system "streams" and "assets" - PID)
Chat (On Load balancer only - URL - http://<host>:8090/juggernaut.js)
Application Health Checks (On App servers only - PID: Passenger, URL - http://www.<host>*/about)
BBB  - API URL

With TCP ports

NODE - TCP - 8090
redis - TCP - 6379 
Apache- TCP - 80
Tomcat - TCP - 8080
mysql - TCP - 3306