[FOSDEM 2008] Programming with Linux on the PS3

2008-08-02T16:20:06Z

On February the 23rd I had the opportunity to give a presentation on programming the Cell BE processor of the Playstation 3 at the 2008 edition of the FOSDEM in Brussels. You can download my slides as pdf or open document format (odp). The talk was recorded by the devroom organizers. The following player can be used to preview it or you can grab the ogg/theora+vorbis video and use a suitable player (such as vlc).

Fosdem2008-olivier-grisel-linux-ps3

| View | Upload your own

If you have a PS3 and want Ubuntu Linux installed on it, please have a look at the official Ubuntu help page.

FYI I had installed the latest version of 7.4 for PS3 using the alternate CD (text based) installer and then upgraded to 7.10 later with aptitude. Other distros have their own PS3 howto.

Once you have Linux up and running I suggest you to first have a look at the following resources:

General documentation for Linux on the Cell by Geoff Levand on kernel.org
Tutorial to learn SPE programming on kernel.org too
The official IBM tutorial
The IBM developerworks resource center that has further pointers to official documentation, forums, SDK releases and third party websites

You can also join the IRC channel #PS3Dev on freenode.org where I tend to be connected as ogrisel.

Finally you might want to have a look at the simd4libsvm project I started on launchpad to add SIMD and threading support to the libsvm open source library. This library is the reference implementation of the Support Vector Machines machine learning algorithms.

How to broadcast a live video stream

2008-06-21T21:31:27Z

As a member of the French Python user group (AFPy) I took part in the organization the second edition of Pycon FR in Paris last week. The event itself was a great success with more than 120 physical attendees and on average 40 people remotely attending the event through our live video stream.

The following shows how we got the stream up and working hoping that our experience might help other benevolent conference organizers to set up a similar 100% free software based infrastructure.

The setup

The Agora room of the Cité des Sciences et de l'Industrie features sonically isolated monitoring room with a high bandwidth internet connection. The room is equipped with 3 remote controlled cameras and 2 microphones. Three monitors and a remote control allows the operator to adjust the cameras and select the one to broadcast. The audio-video analogical signal is then plugged into a movie box that outputs a live DV stream over a firewire link.

The movie box is then plugged into a Core 2 Duo laptop running Ubuntu Gutsy. The laptop is used to encode the DV stream into a ogg theora / vorbis stream using the ffmpeg2theora utility. The unix 'tee' command is used to pipe a local copy of the video on laptop hard drive while broadcasting to the remote streaming server using the oggfwd tool.

The streaming server is a single kimsufi box running Ubuntu Feisty with icecast. The bandwidth of the server is 100 Mbps which should be enough for up to 400 simultaneous clients (for a stream at 250 Kbps). For more clients it is possible to setup several icecast relays and spread the load across them.

Here is the helper script I saved as ~/bin/oggbroadcast on the laptop that did the live acquisition of the DV signal through a cheap PCMCIA / firewire adapter I found here.

#!/bin/sh

# encoding and broadcasting settings
X=320
Y=240
SERVER_NAME=oliviergrisel.name
SERVER_PORT=8000
PASSWORD=theprecious
MOUNT_POINT=/pycon_fr_2008_live.ogg
STREAM_TITLE="Pycon FR live stream"

# local copy settings
BACKUP_DIR=/home/ogrisel/streams
TIME_STAMP=`date +%F-%T`

# ensure the backup directory exists
mkdir -p $BACKUP_DIR

# the actual grabing / encoding / broadcasting chain
dvgrab -f raw - | \
  ffmpeg2theora -a 1 -v 6 -f dv -x $X -y $Y -o /dev/stdout - | \
  tee $BACKUP_DIR/stream_${TIME_STAMP}_${X}x${Y}.ogg | \
  oggfwd -n "$STREAM_TITLE" $SERVER_NAME $SERVER_PORT $PASSWORD $MOUNT_POINT

These encoding settings (X=320, Y=240, -a 1 and -v6) give a 250kbps ogg stream (around 180kbps for the video and 70kbps for the audio). I think these settings are optimal for live streaming since any broadband user should be able to track the presentations without interruption.

The video quality is unfortunately too poor to make it possible to read the slide contents on most presentations. Next year we will collect the slide files earlier to be able to publish them on our website before the beginning of the event so as the remote attendees not to be penalized by the video quality.

Lessons learned

Icecast is a very stable and easy to setup streaming server. I just aptitude install'ed it then tweaked the configuration file under /etc/icecast2 mainly to adjust the passwords for the admin interface and the stream source.
the automatic CPU frequency scaler can get really annoying. On the first day the encoding chain was interrupted several times because for some reason it decided that running at 100% at 1Ghz on one of the cores was better than running at 20% at 2.17GHz. At 1GHz the encoding is not fast enough to make it live without interruption. To avoid those problems set the CPU energy policy to "performance":
```
sudo cpufreq-selector -g performance
```
the quality is good enough for live broadcasting but it would have been nice to put the 'tee' command responsible for making the local copy one step earlier in the chain (before the encoding step). Unfortunately storing the original DV signal takes an awful amount of disk space. Next year it might be a good idea to invest in a new external drive for that purpose only.
advertising the IRC channel name on the stream web page was a very good idea. The internet attendees could exchange together on the presentation hence getting the feeling to really attend the event as almost first class citizens. Some of the geeks in the real life conference room were connected too and could thus discuss about the presentation without disturbing the audience and proxy remote IRC questions to the speaker.

Going further

For the next year I plan to further have a look at other ogg theora streaming tools that might provide additional features:

boxstream, a python graphical utility based on gstreamer to upload a compound stream with overlays to an icecast server
flumotion streaming server, a complete python-based streaming solution (with a server part based on twisted)

By the way, if you speak french, please have a look at the videos (postprocessed by ccomb) and the slides of the presentations.