• 2 x Application Servers
• 2 x GlusterFS storage Servers
• 2 x Mysql Servers
• 1 x Software Load Balancer (Haproxy with SSL Termination)
• 1 X Asterisk (VoIP-SIP-Server)
• 2 X RTMP servers
• 1 X Meeting and Recording Server
• 3 X PVFS (BeeGFS) Servers

Monitoring Servers are self-hosted and based on Cloud.

What Distributed Storage Solution to choose ?

  *  Currently the best bet is to choose GlusterFS for self management. As it has proven record in our current production site.
  *  The other Alternatives are S3 (Object Storage) or EFS (NFS). However it is important to notice that EFS is still in Preview state. Whereas S3 is HTTP Rest API based.
  *  I have shared the S3 Ruby on Rails API and S3 file size limit (5GB limit) details with Uday 
     S3 RoR API - http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/S3.html
     S3 File size limitation - http://stackoverflow.com/questions/10801158/how-stable-is-s3fs-to-mount-an-amazon-s3-bucket-as-a-local-directory
     S3 Performance Considerations - http://docs.aws.amazon.com/AmazonS3/latest/dev/request-rate-perf-considerations.html
     However Streaming performance may not be optimal as S3 is based on HTTP, however need an PoC to check it.

What has been improved in the above architecture ?

   * Highly scalable Video Streaming
   * Improved Hardware specification.

What can be improved ?

  * Storage is given highest priority, however solution was provided above (GlusterFS with BeeGFS or S3 for now and later completely migrate to EFS).
  * Load balancing APP server via Haproxy (loadbalancer) has a bottleneck, which is architecturally similar to ExpertusOne. The solution for this issue is to use Loadbalancer with "Direct Routing" which AWS ELB is based on.
  * Since Our loadbalancer aka Haproxy handles SSL traffic and runs Chat application, its may be difficult to move away as we have already added SSL changes code. This is added to pipeline for future architectural enhancements.

What is currently in the pipeline to optimizing the architecture;

  * Auto scaling using ELB will break the HTTPS and Chat Loadbalancer implementation. Since using Haproxy adds network bottleneck. Trying to find a solution.
  * Automated AMI (Amazon Machine Image) creation after every deployment per region (AWS CloudFront) if required.
    

• Custom scripts should be written to add APP server and to report to Software Load Balancer once it comes up.
• Auto scaling, policy should not rollback/scaledown server until videos are properly processed or Gracefully stopping Service Health monitoring tool “God” from further processing files.
• Find Better alternative to Rsync for realtime delta synchronization or Mount PVFS on APP servers and copy the same files twice (programmaticly via rake task).
• Choosing better AWS zone where newest technologies are implemented first. Suggest AWS Oregon Zone.
• Solve Software Load Balancer network bottleneck to APP servers, Using AWS ELB will break centralized SSL termination.
• We are stuck with Primary storage server as GlusterFS due to our requirement, this issue will be solved once we have AWS EFS ready for production use. Currently AWS EFS is exclusively available in AWS Oregon Zone.