===== WebCam Recording =====
===== Requirement =====
One should be able to record one self on a webcam and create a content which can be added to a course.
===== Implementation =====
- The webcam stream is recorded using a flash client.
- It is published to red5 media server latest version(1.0 RC1) through rtmp and saved.
- The crossbow app copies the file into local public directory and creates contents from it.
===== Components and repository locations =====
**1. Recorder client -**
This is a small flex3 app.
Located at - https://repos.exphosted.com/svnrepos/collaboration/recorder/client/trunk
We need not deploy it anytime. It is compiled to a swf file which is placed in crossbow app's repository in public/recorder.
This moves out along with crossbow app.
**2. Steps to convert the recorder client flex project into swf file -**
a) In flex developer add the following argument in Additional compiler arguments text field:
-theme=${flexlib}/themes/Halo/halo.swc
b) Download the following player swc file:
http://fpdownload.macromedia.com/pub/flashplayer/updaters/11/playerglobal11_0.swc
c) Move it to the below flex sdk location:
/frameworks/libs/player/11.2/playerglobal.swc
Right now the app is built on flex sdk 4.6 and Flash Player 11.0
**3. Recorder server -**
This is a red5 service written in java.
Located at - https://repos.exphosted.com/svnrepos/collaboration/recorder/server/trunk
We need to deploy it to red5 server whenever we make any changes. The red5 setup and deployment changes are below.
** Integrating with crossbow application **
1. The 'red5recorder.swf' generated from the build of the flex app is placed in the public directory of the crossbow app.
2. The rede server app runs on a dedicated machine( may be on the machine with bbb server). The configuration details are to be updated in recorder.yaml file of crossbow app.
3. Embed the red5recorder.swf file on the page needed to start recording.
===== Setting UP Red5 server app =====
==== Install red5-1.0 RC1 =====
If you already have red5 setup you could skip this step. Red5 server would have been already setup on machines with Bigblue button. So, no need for this setup. But, Please make sure that the red5 server 1.0 version is installed.
Red5 requires java jdk 6. Intstall it if not installed already.
If they aren’t already installed, add the following to /etc/apt/sources.list
deb http://us.archive.ubuntu.com/ubuntu/ lucid multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ lucid multiverse
deb http://us.archive.ubuntu.com/ubuntu/ lucid-updates multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ lucid-updates multiverse
Now, install java
apt-get update
apt-get install java-package
apt-get install sun-java6-jdk
apt-get install sun-java6-jre
Now, install red5 server **latest version**
wget http://trac.red5.org/downloads/1_0/red5-1.0.0-RC1.tar.gz
tar xvfz red5-1.0.0-RC1.tar.gz
mv red5-1.0.0 red5
mv red5 /usr/share/
Now, start the red5 server
cd /usr/share/red5
sh red5.sh
===== Developing a red5 application =====
Red5 is an open source media server for flash content.
The resources about red5 server can be found below
* Home page - http://www.red5.org/
* Documentation - http://trac.red5.org/wiki/Documentation
* A deatiled screen cast session for an absolute beginner - http://www.red5guide.com/
These screencasts give you a nice demo of how to build a sample red5 app. These are given for a windows machine. They should work on ubuntu also in the similar lines.
===== Building the recorder server app and Deploying it to red5 server ====
Make sure that the machine has **red5-1.0** server. Usually it is located in /usr/share/red5.
If you have a red5 version which is earlier than 1.0 RC1 please upgrade it. Find the instructions above to install red5 server.
Also, make sure that the user has read, write and execute permissions to the directory /usr/share/red5/webapps.
We need ant to build the code. Install it if not installed already.
apt-get install ant
1. Check out the code of recorder server to a local directory - https://repos.exphosted.com/svnrepos/collaboration/recorder/server/trunk
svn co https://repos.exphosted.com/svnrepos/collaboration/recorder/server/trunk recorder
2. go to recorder and run ant and ant -p
cd recorder
ant
ant -p
3. You should see the builds complete successfully and a .war file in recorder/dist directory.
It should be recorder/dist/recorder.war
Now , the server is built successfully.
To deploy the war file, copy it to your red5 server webapps directory. Usually it would be at /usr/share/red5/webapps
cp -r recorder/dist/recorder.war /usr/share/red5/webapps
**Note :** Please remove the file recorder.war and directory recorder if they already exist in the location.
Red5 server has a war deployer service which typically runs every 10 mins. So, in less than 10 mins you should see a directory "recorder" is created.
Restart the server if needed.
===== Local directory setup for crossbow app ======
The webcam recorder streams will be saved in the directory /usr/share/red5/webapps/recorder/streams on the machine where the red5 server is running. (Usually the same machine as that of BBB). The crossbow app could be running on another machine/cluster. When the user saves the details of document, we want to copy the file from the streams directory to the public directory of crossbow app and save it.
So, One way to have this to mount the steams directory of BBB machine on the machine where crossbow is running and specify this location in the crossbow app's config/recorder.yml file. ( I am not sure if rsync is the best one for clustered environment.)
The recorder.yml(https://repos.exphosted.com/svnrepos/crossbow/trunk/config/recorder.yml) file has two settings for an environment
server_url: rtmp://127.0.0.1/recorder
stream_base_directory: /share/recorder_data/streams
1. Replace the IP address in the "server_url" attribute with the IP address of the machine where the red5 server is running.
2. Replace the "stream_base_directory" value with the local mount location of "/usr/share/red5/webapps/recorder/streams" directory of the red5 server machine.
3. Please make sure that the steams directory is readable and writable by the rails app process.
**Verifying the streams stored location**
Please make sure that when we record the application, the videos are being saved to the "/usr/share/red5/webapps/recorder/streams" directory of the application. This can be tested by launching the crossbow app and create a video content which is recorded.
If there is any issue with saving/detecting the server please change the red5.properties "proxy.destination_host=127.0.0.1" to
"proxy.destination_host="
the red5.propeties file can be found at - /usr/share/red5/conf/red5.properties
Restart the red5 server and make sure both recorder and BBB work fine.
===== Video Quality discussion ======
At this moment the camera is sending data at 15 frames per second. This is a reasonable level of quality considering bandwidth limitations and server capture capacity. This can certainly be made to something around 24-25 frames per second to achive a high quality
video. But, when recorded from a slow bandwidth machines it may cause drag in the video frames.
This is passed as a parameter to setMode function
cam.setMode(400,300,25);
Also, one more setting to specify quality is available on a webcam. This specifies whether importance is given for quality or bandwith.
first param specifies the priority of bandwidth and the second is that of quality on a scale of 1-100. We do not use it as of now.
cam.setQuality(0,100);
The quality of the web cam video is dependent on webcam settings, internet speed and the encoding used by the flash player to send the videos. Added an option setKeyFrameInterval to the camera and also made sure that the camera settings are reasonable.
Will have to do more study on increasing the stream quality sent by flash if we need higher quality videos. May be, we can use - http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC [^] some time in the future to improve this.
===== Video Stream Quality ======
1. The generated streams quality is less(videos are pixelated) and also the audio is out of sync with the video. This is a common problem with current recording using red5, especially on slower internet speeds with 640x480 video sizes. The following link explains the problem and points some solutions.
http://avchathq.com/blog/recording-high-quality-flash-video-over-slow-internet-connections/
The following is the summary of our trials with various options.
^ Tip ^Recommended ^ Notes ^
| Buffering videos from client | yes | Keeping the buffertime on the client improved the quality by of the video by a good amount.With red5 this did not help me a lot on slower bandwidths when tested on dev01. With red5 when we buffer the video quality is becoming low and some times we do not even see a video.|
| Change red5 config | yes | Need to in crease queueThreshold to a higher value like 2400 to tell red5 to wait for more time before writing data to flv file. This looks good on local but on collabdev01 this was of not much help.|
| Other Media servers (Wowza3 or FIMS4) | | Tried a sample with Wowza. The voice sync issue is not observed. Quality os also be much better but it improved more after making some changes. FMIS 4 also looks much better in terms of the quality of the video. The video file sizes are more than what we get with red5 server.|
| Video conversion with ffmpeg or xuggle | | To be tried |
| Video encoding(h.624) | | To be tried|
==== Observations after testing on dev01 on March 14 ====
^ Media server ^Client buffers ^Internet speed ^ Video Quality ^Voice out of sync issue ^
| Red5 | yes | decent | very poor(pixelated) and random. Some times only blank screen comes up | not present |
| Red5| no | decent | poor and random. | present |
| Wowza| yes | decent | very good. | not present |
| Wowza| no | decent | poor and strucks for a few seconds initially. Much better than that of red5. | not present |
| Red5 | yes | low | very poor(pixelated) and random. Most of the times only blank screen comes up. No video found. | not present |
| Red5| no | low | poor and random. The video stuck at many points. | present |
| Wowza| yes | low | good but struck for a few seconds initially | present on very low speeds. |
| Wowza| no | low | poor and pixelated. Some times it struck. | present |
Videos were recorded for about 20-25 seconds.
Low internet speed is tested from a wifi connection.
Decent internet speed is observed from wired connection in the office.
Client buffers -
* 'yes' is the client from recorder trunk - revision 91.
* 'no' is the client from the latest revision till date.
===== Interaction Flow ======
{{:recording_flow.png|}}
====== Webcam Recording - Revised ======
The methods tried above were not sufficient to provide usable and production ready recordings due to the following limitations.
1. We wanted 640x480 recording which is considered high quality for flash player based recording. Flash player default video encoding is not good. They had high bits per second value which demanded high bandwidth for users and flash player < 11.0 version can record videos upto 1 mbps speed only.
2. The stream buffering techiques of flash player do not really work as expected.
3. Red5 is not very good when it comes to recording and recording is broken with version 0.91 which caused more problems.
===== Solution =====
1. As flash player 11 supports client side video encoding to h264, video size is considerably reduced which saves bandwidth.
2. Red5 1.0.2 and 0.8 versions have a decent support for recording. But recording never worked smooth when both audio and video are combined into a single stream.
So, considering the above, we can take this following approach.
1. Webcam stream and microphone audio are recorded separately as individual streams by two different net connections. Red5 1.0.2 is used.
2. They are merged together using ffmpeg to make a single video.
3. Mic and camera are going to have a sync issue as they are separate streams. This sync is fixed by calculating the difference and adjusting it with ffmpeg.
4. However streams may some times have sync issue of < 0.5 seconds which is acceptable for usability. This is also yhe case even if we recording mic and cam to a single stream due to the limitation in the recording rate of devices.
5. **Codecs** - webcam video is recorded as h264 within a flv container.
- Micrphone audio has two options, nellymoser and speex. For now, we are using nellymoser codec as the gain from speex codec for the same quality is around 10kbps only.
===== Changes needed ======
1. Webcam recording client needs to be completely rebuilt to record two streams separately.
2. Changes on crossbow app to integrate this new webcam client and ffmpeg code to merge the videos. The webcam videos are to be processed and merged in background.
3. red5 1.0.2 server needs to be setup on all environments. Recorder server app can be deployed to this red5 server.
==== Merging audio and video streams ====
Use ffmpeg to merge audio and video streams recorded. Say audio file is record_audio.flv and webcam video file is record_video.flv
STEP 1 - Remove the additional length of the audio file. Always observed that audio is longer than video as mic sends data faster initially than webcam. So, find the lengths of the audio and video files and clip the difference time from audio from the beginning.
#say 2.5 is the differnce b/w audio and video lengths (in seconds).
ffmpeg -i record_audio.flv -ss 2.5 -t 100000000 recrod_audio_adjusted.flv
STEP 2- Merge the video file and adjusted audio file together and trans-code as needed.
ffmpeg -i record_video.flv -i record_audio.flv -c:v copy -c:a libfaac -f mp4 record.mp4
===== Deployment instructions =====
1) On Collab machine - Point the streams directory of red5 'vod' application to shared streams and change permissions
cd /usr/share/red5/webapps/vod
sudo mv streams streams_backup
sudo ln -nfs /usr/share/red5/streams streams
sudo chown -h red5:red5 streams
2) On Collab machine - Change the queue threshold setting value of red5 server to a high number.
sudo cp /usr/share/red5/conf/red5-common.xml /usr/share/red5/conf/red5-common_backup.xml
sudo vi /usr/share/red5/conf/red5-common.xml
#change existing queueThreshold property value to 7200000. The line looks like below.
3) on Crossbow app server - Change the settings in the recorder.yml config file of crossbow rails app.
vi /deploy/crossbow/shared/config/recorder.yml
#update the values here..
:
video_server_url: rtmp:///vod
video_stream_base_directory: /deploy/crossbow/shared/streams
audio_server_url: rtmp:///recorder
audio_stream_base_directory: /deploy/crossbow/shared/streams
4) Post Deployment: Run the following rake task (yet to run on dev)
rake backfill:create_video_recording_real_time_conversion_duration_setting
===== Known issues =====
1) As we are using red5 0.91 of BBB now, this may have issues recording long videos. With our testing 2 out of 6 videos > 30 mins had issues where some part of the recording is missing. This issue is not present with red5 1.0.2. So, moving to red5 1.0.2 is highly recommended in the future.
2) For some recordings there may be a voice and video out of sync for <= 0.5 seconds which are still very much usable. This is a rare issue and is due to the difference in mic and cam rates and not possible to fix this consistently for all videos. As this the videos are still usable and clean, this is fine. This issue is also observed with Flash media server some times.