======= Streaming videos with jwplayer and Red5 (And other media servers) =======

We should find out a way to stream our videos with the jwplayer as progressive download will be little slow when some one watches the video the first time.

===== JWplayer and red5 =====

JWplayer supports sreaming with rtmp out of box. The details can be found - http://www.longtailvideo.com/support/jw-player/46/streaming-video-with-the-jw-player. 

jwplayer does not support red5 server officially but it worked fine when we tried.

=== Code Changes ===

1. Once a video is uploaded and processed, it is copied to a location from where red5 can stream. For now, we are using recorder as the streaming end point and we copy the videos to a a directory in recorder/streams.

2. When we are playing, if a video is present in the streams directory then stream it otherwise fallback to progressive download from crossbow content processed directory.

=== Configuration change for crossbow ===

We can use the streaming settings in streamer.yml file.

<code>
development:
  server_url: rtmp://devcollab01.exphosted.com/recorder
  base_directory: /deploy/crossbow/shared/streams
  uploaded_path: uploaded  #This is the directory or path inside "base_direcory" where videos get saved
</code>


=== Observations ===

1. Streaming improved the speed clearly when the video is being watched for the first time. Tried with a video of size 80 MB and 15 mins length with 3 people watching the video at the same time.
   - The __stremaing looked smooth__ on dev01 with red5.
   - There is a bit of lag when we seek the player but still the overall experience is good.
   - The __experience is slightly better than that i observed with wowza__ on wowza01.exphosted.com
   

===== JWplayer and Wowza =====   

Tried the streaming with the trial of wowza on wowza01.exphosted.com. Configured the videorecording example of wowza to stream the videos. The rest of the code is same as that of red5 mentioned above.

=== Observations ===

1. For the above mentioned video file, with wowza the streaming experience is good but still red5 is a little better and smoother.


===== To Be Tried =====

1. Need to test the streaming under high traffic where 100s or atleast 10s pf people are watching videos. (this cuases multiple connectons of red5 server. Need to see the performance of this.). Also, try out if any load balancing can be if required.

2. To stream through firewalls where rtmp port is blocked, jwplayer falls back to rtmpt and uses port 80. Need to see if any changes are required on red5 side. 
   

===== Open Question =====

1. Streaming with jwplayer always sends a request to media server and starts playing the video from the server. If we use progressive download, the video is downloaded for the first time and from next time it is served from the browser's cache (before we clear the cache or end the session). Is there any trade off we can make on which videos to stream and which ones to download to get the best user experience ?
   

===== Managing Streaming from crossbow =====

==== Enable streaming (say with red5)  - ====

1. Create a directory named "uploaded" (or any filename you like) inside /deploy/crossbow/shared/streams (the location to which red5 recorder streams directory is mounted)

2. Specify the following configurations in crossbow/current/config/streamer.yml
<code>
<RAILS_ENV>:
  server_url: rtmp://127.0.0.1/recorder #the recorder (or any red5 app) end point url
  base_directory: /usr/local/red5/webapps/recorder/streams # the location to which red5 recorder streams directory is mounted
  uploaded_path: uploaded # the directory name we have created in step 1
</code>

3. To copy already uploaded video files to streams/uploaded directory

Please run this task in that particular environment
 
   rake content:copy_videos_to_streaming_server


==== Disable streaming - ====

- If you like to remove all the streams from the media server, just remove them manually or run the following task. 

rake content:delete_videos_from_streaming_server

This removes the files from the media server's uploaded directory as configured in streamer.yml file
   
- If we like to disable streaming please the server_url entry to blank. This does not buffer videos but still copies newly created videos to media server's shared directory. To stop this, please set base_directory and uploaded_path to blank.

** Note - ** Please do not delete streams.yml file. Just remove the values for the yml file entires in that particular environment.


==== Switch streaming to a new server - ====

Please follow the steps in enable streaming but just change the streamer.yml file entries to what we want.

** Note - ** Every time you suspend the streaming and re enable it just set the configurations and then run content:copy_videos_to_streaming_server task so that if any videos are created and not copied to media server, they will just be copied.

===== Setting Up red5 and An application =====

We may need to install red5 server and have at least one red5 app installed to make the streaming work (you can use any demo app or setup our recorder app)

To install red5 - https://wiki.exphosted.com/doku.php/webcam_recording#install_red5-10_rc1

To setup recorder app - https://wiki.exphosted.com/doku.php/webcam_recording#building_the_recorder_server_app_and_deploying_it_to_red5_server

If you like to use an example app of red5 for stremaing (say oflaDemo)

1. Create a direcory called streams inside <Red5_HOME>/webapps/oflaDemo. Create a directory named 'uploaded' inside streams.

Now you should have this directory structure - <Red5_HOME>/webapps/oflaDemo/streams/uploaded
   
2. Now, change the streamer.yml entry to something like below
<code>
<RAILS_ENV>:
  server_url: rtmp://127.0.0.1/oflademo
  base_directory: <red5_HOME>/webapps/oflademo/streams
  uploaded_path: uploaded
</code>

===== Video Quality and bitrate for streaming =====

Bit rate is number of bits of data(audio + video) per unit time. In general, measured in bits/second.

When the bit rate of the video being streamed is more than the available bandwidth the video streaming does not happen smooth, as the data to be downloaded is more than the available band with at any point of time.

**What can cause issues with bit rates with the current code ?**

With the current code, we convert uploaded videos to flv format using ffmpeg and we have an option qscale with a lowest value(0.1) to maintain the quality high. This can cause increase in the bit rate (and the overall file size) for some videos which are of good quality already. 

**What are the suggested bit rates used for videos that can be streamed ?** 

- For standard videos we can use 300kbps to 600kbps
- For HD videos we can use 700kbps to 1200kbps

//Note -// Could come up with this number after referring the following links. 1800kbps seemed to be a bit high. So, ignored it.

http://stackoverflow.com/questions/604390/flash-encoding-flv-what-are-the-prefered-bitrates
  
http://www.learningapi.com/streamingmedia-articles/dynamic-streaming-in-flash-bitrate-switching/

** Can't we keep the bit rate low and also retain the quality ? **    

This is not possible with all the video formats we are using. This is highly dependent on the video codec. A mp4 video cannot give the same quality of a mp4 video without change in the file overall size (i.e also the bit rate). We should figure out the best possible video format for streaming so that the bitrate is low for a given quality. With jwplayer and red5 combination it is either flv or mp4.
    

==== Possible Fixes ====

We need to improve the video conversion process and also the way we pick up a video to play.


=== 1. Bitrate switching with video player ===

We should have multiple videos with multiple bit rates and sizes and show the best possible video to the user based on his bandwidth, so that the streaming is smooth. This is the best possible solution and being used widely by lot of video streaming apps. 

(Bitrate switching with jwplayer - http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v4/12208/video-delivery-rtmp-streaming#dynamicstreaming)

  * I think we can go with 4 versions of (video) bit rates -> 256kbps, 512kbps, 768kbps, 1024kbps
  * Generally the audio bit rate can be kept a constant value 128kbps for decent quality.
(If we need to support higher quality and higher bit rates we can also consider 1800kbps as highest quality)
 
The standard width of the player is 640x480. So, 256kbps, 512kbps, 768kbps videos can be of size 640x480. The highest quality video dimesnions can be something like 1200x1080.

== Open issues ==

1. Need to try out if all the videos should have all the 4 bitrate versions or there should be a trade off based on some parameters.

== Note == 

We have not implemented bit rate switching in the product yet. 

=== 2. Improvements during video processing ===

== 1. ffmpeg option changes ==

We should try to get the options for video conversion so that the quality is as good as the source video and the bit rates are as small as possible.
The following things should be tried out

__* Resize videos to suit our player and full screen mode.__

Re sizing videos to 640x480 reduces the bit rate as the size of the video is reduced. The quality remains same. This should be done.

== Open issue ==
  
If the size of the video is not exactly 640x480, black borders may appear around the video as it tries to adjust. We should have the video player sized exactly to 640x480 to avoid black borders around the video.
  
__* Qscale and Bitrate options__
  - Specifying a qscale option overrides the bitrate. Low qscale value results in high bit rates. High qscale value resulted in low quality video.
  - qmin, qmax values reduce bit rate but the quality of the video degrades based on the original video's quality.
  - An option to specify a maximum bit rate would have been ideal but this does not seem to work with ffmpeg. Tried to use minrate and maxrate options. They did not help much to keep the quality.


== 2. Best video format for streaming ==

With jwplayer and red5 steaming to work, we should pick one of the two - flv or mp4.

- flv is working smooth during steaming but resulting a high bitrate.
- mp4 results in low bit rates, but some pf the processed mp4 did not stream from red5. We need to figure out if this is fixable.

After several tries, mp4 videos with videos encoded with h.264 and audio with AAC has the best quality for a given bit rate.

But some flv videos and 3gp videos may have incompatible audio/video codecs to convert them to mp4(vcodec h.264 and acodec AAC). Hence we removed the support for 3gp and the flvs are currently handled in the same format (flv).

One more advantage of going for mp4 videos is that it is supported by HTML5 video tag on all the browsers except FF and Opera. With a combination of mp4 and webm we can cover all the browsers in HTML5 mode.

== 3. ffmpeg alternatives for video conversion ==

Looked at some of the opensource alternatives for ffmpeg. Most of the video conversion tools are based on ffmpeg. One tool which looked very nice was - http://handbrake.fr/. This generated mp4 videos with good quality and less bitrates. But we cannot use this as the out put format is only mp4 and this does not do re sizing of the video well.

== Conclusion ==

We use ffmpeg to genrate mp4 videos with an average video bitrate of around 512k and audio bit rate of 128k for a reasonable streaming.
The video is encoded with h.264(AVC) and audio with AAC. Tried this with sample videos of the formats f4v, mp4, m4v, mov, mpeg and the comversion worked fine.

== Discussion ==

* h.264 compression is a heavy process in CPU and the video conversion to mp4 may take a little longer than conversion to flv. But, this should not be an issue for now. When we support really lengthy videos this process can be back grounded.