RTSP-java.net

Since my book on Mobile Media API (MMAPI), <a
href="http://www.amazon.com/gp/product/1590596390/">Pro Java ME MMAPI: Mobile Media API for Java
Micro Edition, was published in May, I have been inundated with requests to help readers
with streaming content via MMAPI for Java-enabled mobile devices. This topic was an important
omission from the book, but one that was simply not feasible to include because of the lack of
support for it within various MMAPI implementations. In this article, I will show you the results of
experiments I have conducted since the publication of the book to stream content via MMAPI using a
custom datasource.

DISCLAIMER: Before I commence, I would like to point out that even though I was
able to stream data from a streaming server and receive it successfully in a MIDlet using a custom
datasource, I wasn't able to utilize this data in any meaningful manner because of limitations in
the way this data is read by the MMAPI implementation at my disposal. You may have more success if
you have access to a MMAPI implementation that doesn't read its data fully. Even if you don't, this
article still provides a good study of the issues involved in streaming media data. At the very
least, it shows you how to create and utilize your own custom datasource.

For a background on Java ME please see my previous <a
href="http://today.java.net/pub/au/179">tutorial series on getting started. For an introduction
to MMAPI, tutorial 4 is a good
start, or you can always buy the book.

Background to the streaming problem

MMAPI is a format- and protocol-agnostic API, which means that the API doesn't dictate mandatory
support from device manufacturers for any particular format or protocol. One of the protocols that
is widely requested by application developers is the <a
href="http://en.wikipedia.org/wiki/RTSP">Real Time Streaming Protocol (RTSP) and the
associated Real-time
Transport Protocol <a
href="http://en.wikipedia.org/wiki/Real-time_Transport_Protocol">(RTP) for streaming audio/video
content. The advantage of streaming content is that it provides a fast turnaround time for the user,
control over the content distribution to the distributor, and an overall richer user experience.

However, hardly any manufacturer supports this protocol through Java ME. Some new phones provide
support for RTSP, but that support is only on a smattering of devices. A majority of devices still
do not support this protocol, therefore limiting useful application development in the streaming media
department. A majority of questions in the MMAPI forums of various device manufacturers revolve
around this very issue, that is, how to provide streaming data when RTSP is not supported. This
article aims to point you in the right direction. I'll start by cutting through the
clutter to try to provide an understanding of what streaming means.

What is streaming?

Streaming is the process of transferring data via a channel to its destination, where it
is decoded and consumed via the user or device in real time, that is, as the data is being
delivered. It differs from non-streaming processes because it doesn't require the data to be fully
downloaded before it can be seen or used. Streaming is not the property of the data that is being
delivered, but is an attribute of the distribution channel. This means, technically, that most media
can be streamed.

HTTP and RTSP

HTTP and RTSP are application-level protocols that allow remote retrieval of data. So why
can't you use HTTP for streaming media content? The truth is, you can. When you click on a Web page
link to play an audio file, in most cases the media data is streamed to your machine. However,
streaming content over HTTP is inherently inefficient. This is because HTTP is based on the <a
href="http://en.wikipedia.org/wiki/Transmission_Control_Protocol">Transmission Control Protocol
(TCP), which makes sure that media packets are delivered to their destination reliably without
worrying about when they are delivered. On the other hand, RTSP can be based on both <a
href="http://en.wikipedia.org/wiki/User_Datagram_Protocol">User Datagram Protocol (UDP), which is
a connectionless protocol ensuring faster delivery over reliability, and on TCP. Besides, RTSP has
control mechanisms built in that allow random access to the media data, allowing you to seek, pause,
and play.

Making sense of RTSP, RTP, and RTCP

There is a lot of confusion among newcomers over the acronyms RTSP, RTP, and RTCP. All three
represent different protocols related to streaming of media content. An RTSP session initiates both
Real-time Transport Protocol (RTP) and RTP Control Protocol (RTCP) sessions. RTSP is only the
control protocol, a bit like a remote control for a DVD player, in that it allows you to start,
stop, resume, and seek data remotely. The actual data delivery is done via RTP, and RTCP is a partner
protocol to RTP providing feedback to both the sender and receiver on the quality of media data that
is being transferred.

With this basic introduction about RTSP and streaming out of the way, let's set up our own
streaming server to conduct some experiments. You can read more about RTSP, RTP, and RTCP at
http://www.rtsp.org.

Set up a streaming server

To conduct experiments for the purposes of this article, you will need access to a specialty
streaming server that can create RTSP streams for media objects. One such server is the <a
href="http://developer.apple.com/opensource/server/streaming/">Darwin Streaming Server, which
is an open-source streaming server based on the same source code as Apple's commercial QuickTime
streaming server. Implementations of this free server are available for Mac OS, Linux, and Windows.
Download the version that is suitable for your OS and run the installer. You can also choose to
download the source code and build it in your environment. I have run the examples in this article
on a Windows XP machine, and the server is installed in C:\Program Files\Darwin Streaming
Server .

For the purposes of this article, you will also need to have Perl installed on your computer, to
administer the Darwin server. For Windows, you can download <a
href="http://www.activestate.com/Products/ActivePerl/">ActivePerl.

As part of the installation, you will be asked to provide an administrator username and password,
but make sure that you run the administration server after the installation (by running the
streamingadminserver.pl file). This starts an administration server on port 1220 with which
you can monitor the current activity within the streaming server. More importantly, you will need to
supply a username/password combination the first time you log into the administrative console (by
navigating to http://localhost:1220 in your browser) for running the movie and MP3
broadcast service. It is important to set this (even though you never really need to supply this
username/password combination anywhere when running the examples in this article).

Note: On Windows, if you download the latest version of ActivePerl,
streamingadminserver.pl is likely to fail with the following error:

ActivePerl 5.8.0 or higher is required   in order to run the Darwin Streaming Server web-based administration. Please   download it from http://www.activeperl.com/ and install it.

This is because of an incorrect configuration check in this script, and you can easily fix it by
commenting out lines 33 and 34 (put a # in front of these lines).

The streaming server starts on port 554 and comes with a few sample movie files, ready for
streaming in the installation folder under the Movies directory. The Darwin server can stream MPEG-4,
3GPP, and QuickTime movie files natively. This means that these files don't need to be "hinted" in
order to be streamed. Hinting is a process by which media files are prepared with track
information for streaming using the professional version of <a
href="http://www.apple.com/quicktime">QuickTime. For the purposes of this article, I will work
with natively streamable files like 3GPP and MPEG-4 only.

To test that your streaming server is working correctly, use the <a
href="http://www.apple.com/quicktime/download/standalone.html">QuickTime player to launch a file
via RTSP. For example, if you can open the URL rtsp://localhost:554/sample_50kbit.3gp
correctly in the Quicktime player and view the file, pause it, stop it, and seek it, then your streaming
server is working correctly.

Model an RTP packet

As I said earlier, RTP is the actual delivery protocol for streaming data. Each streaming session
involves the streaming server sending RTP packets to its destination based on the client request
(requests that are delivered via the RTSP protocol). A full knowledge of the <a
href="http://tools.ietf.org/html/3550">RTP RFC is not required for the purposes of this article,
so the following base class will model an <a
href="http://www.networksorcery.com/enp/protocol/rtp.htm">RTP packet to its best possible
approximation.

Note: I have used the Java ME
Wireless Toolkit 2.3 (beta) to create and run the examples in this article. You can start by
creating a project called "StreamingData" (or whatever you prefer) in this toolkit to place your
code in. The J2ME tutorial part
1 gives more details on the process of creating projects in this toolkit.

public class RTPPacket {  // used to identify separate streams that may contribute to this packet  private long SSRC;  // incrementing identifier for each packet that is sent  private long sequenceNumber;  // used to place this packet in the correct timing order  // that is, where this packet fits in time based media  private long timeStamp;  // the type of the media data, or the payload type  private long payloadType;  // the actual media data, also called the payload  private byte data[];  // the get and set methods  public long getSSRC() { return this.SSRC; }  public void setSSRC(long SSRC) { this.SSRC = SSRC; }  public long getSequenceNumber() { return this.sequenceNumber; }  public void setSequenceNumber(long sequenceNumber)    { this.sequenceNumber = sequenceNumber; }  public long getTimeStamp() { return this.timeStamp; }  public void setTimeStamp(long timeStamp) { this.timeStamp = timeStamp; }  public long getPayloadType() { return this.payloadType; }  public void setPayloadType(long payloadType)    { this.payloadType = payloadType; }  public byte[] getData() { return this.data; }  public void setData(byte[] data) { this.data = data; }  public String toString() {    return      "RTPPacket " + sequenceNumber +      ": [" +      " ssrc=0x" + SSRC +      ", timestamp=" + timeStamp +      ", payload type=" + payloadType +      " ]";  }}

The comments within the code should offer you some idea about the various features of an <a
href="http://www.networksorcery.com/enp/protocol/rtp.htm">RTP packet. Since you won't be building
a complete RTP client and will be running this code within the confines of this example, the main
feature of the above class is the data, or the payload contained within such a packet. Note that an
RTP packet contains other information as well, which is not modeled by this class.

Create a custom DataSource

A DataSource is a MMAPI abstract class, implementations of which encapsulate the
task of media data location and retrieval. Device manufacturers provide their own implementations in
the Java ME toolkit for most sources. Developers don't need to create their own custom datasources
because the task of locating data over file or network is rudimentary and fulfilled by the device
manufacturer's implementation. However, in cases where the developer needs to do data retrieval from
a custom source, a custom datasource is the answer, and media data fetched from a streaming server
is a perfect example.

Data retrieval is one thing, while data consumption is another. Since MMAPI doesn't
allow you to create custom media players, will a custom datasource suffice in this example? Let's
proceed further with the creation of the custom datasource before I answer that question. The
following listing shows the starting of the custom datasource class that I will use for talking to
the streaming server:

import javax.microedition.media.Control;import javax.microedition.media.protocol.DataSource;import javax.microedition.media.protocol.SourceStream;public class StreamingDataSource extends DataSource {  // the full URL like locator to the destination  private String locator;  // the internal stream that connects to the source  private SourceStream[] streams;  public StreamingDataSource(String locator) {    super(locator);    setLocator(locator);  }  public void setLocator(String locator) { this.locator = locator; }  public String getLocator() { return locator; }  public void connect() {}  public void stop() {}  public void start() {}  public void disconnect() {}  public String getContentType() { return ""; }  public Control[] getControls() { return null; }  public Control getControl(String controlType) { return null;  }  public SourceStream[] getStreams() {    return streams;  }}

This class contains only placeholder methods at the moment. Internally, each datasource uses a
SourceStream implementation to read individual streams of data from; therefore,
let's create a simple SourceStream implementation for reading RTP packets:

import java.io.IOException;import javax.microedition.media.Control;import javax.microedition.media.protocol.SourceStream;import javax.microedition.media.protocol.ContentDescriptor;public class RTPSourceStream implements SourceStream {  public RTPSourceStream(String address) throws IOException { }  public void close() { }  public int read(byte[] buffer, int offset, int length)   throws IOException {   return 0;  }  public long seek(long where) throws IOException {   throw new IOException("cannot seek");  }  public long tell() { return -1; }  public int getSeekType() { return NOT_SEEKABLE;    }  public Control[] getControls() { return null; }  public Control getControl(String controlType) { return null; }  public long getContentLength() { return -1;    }  public int getTransferSize() { return -1;    }  public ContentDescriptor getContentDescriptor() {      return new ContentDescriptor("audio/rtp");  }}

As with the previous listing, this class only contains placeholder methods for the moment. However, all listings so far should compile and preverify successfully.

Creating an RTSP Protocol Handler

Recall that RTSP is the actual protocol over which streaming commands are initiated, through
which the RTP packets are received. The RTSP protocol
is like a command initiator, a bit like HTTP. For a really good explanation of a typical RTSP
session, please see these <a
href="http://folk.uio.no/meccano/reflector/smallclient.html">specifications for a simple RTSP
client. For the purposes of this article, I am going to oversimplify the protocol
implementation. Figure 1 shows the typical RTSP session between a client and a streaming server.

<a href="http://today.java.net/images/2006/08/experiments-figure1.gif"
onclick="window.open('/images/2006/08/experiments-figure1.gif','fullsize','toolbar=no,width=553,
height=941,status=no,location=no,scrollbars=yes,resizable=yes,menubar=yes');return false"><img
src="/images/2006/08/experiments-figure1-sm.gif" vspace="4" alt="Figure 1 - A typical RTSP session
between a RTSP client and a streaming server" width="220" height="374" border="0" />
Figure 1. A typical RTSP session between a RTSP client and a streaming server (click for
full-size image).

In a nutshell, an RTSP client initiates a session by sending a DESCRIBE request to
the streaming server which means that the client wants more information about a media file. An
example DESCRIBE request may look like this:

DESCRIBE rtsp://localhost:554/media.3gp rtsp/1.0CSeq: 1

The URL for the media file is followed by the RTSP version that the client is following, and a
carriage return/line feed (CRLF). The next line contains the sequence number of this request and
increments for each subsequent request sent to the server. The command is terminated by a single
line on its own (as are all RTSP commands).

All client commands that are successful receive a response that starts with

RTSP/1.0 200OK

. For the DESCRIBE request, the server responds with several parameters, and
if the file is present and streamable, this response contains any information for any tracks in
special control strings that start with a a=control:trackID= String. The
trackID is important and is used to create the next requests to the server.

Once described, the media file's separate tracks are set up for streaming using the
SETUP command, and these commands should indicate the transport properties for the
subsequent RTP packets. This is shown here:

SETUP rtsp://localhost:554/media.3gp/trackID=3 rtsp/1.0CSeq: 2TRANSPORT: UDP;unicast;client_port=8080-8081

The previous command indicates to the server to set up to stream trackID 3 of the
media.3gp file, to send the packets via UDP, and to send them to port 8080 on the client
(8081 is for RTCP commands). The response to the first SETUP command (if it is okay) will
contain the session information for subsequent commands and must be included as shown here:

SETUP rtsp://localhost:554/media.3gp/trackID=3 rtsp/1.0
CSeq: 3
Session: 556372992204
TRANSPORT: UDP;unicast;client_port=8080-8081

An OK response from the server indicates that you can send the PLAY command, which will make the server start sending the RTP packets:

PLAY rtsp://localhost:554/media.3gp rtsp/1.0CSeq: 3Session: 556372992204

Notice that the PLAY command is issued only on the main media file, and not on any
individual tracks. The same is true for the PAUSE andTEARDOWN commands,
which are identical to the PLAY command, except for the command itself.

The following listing contains the RTSPProtocolHandler class. The comments in the
code and the brief information so far should help with understanding how this protocol handler
works:

import java.util.Vector;import java.io.InputStream;import java.io.IOException;import java.io.OutputStream;public class RTSPProtocolHandler {    // the address of the media file as an rtsp://... String    private String address;    // the inputstream to receive response from the server    private InputStream is;    // the outputstream to write to the server    private OutputStream os;    // the incrementing sequence number for each request    // sent by the client    private static int CSeq = 1;    // the session id sent by the server after an initial setup    private String sessionId;    // the number of tracks in a media file    private Vector tracks = new Vector(2);    // flags to indicate the status of a session    private boolean described, setup, playing;    private Boolean stopped = true;    // constants    private static final String CRLF = "\r\n";    private static final String VERSION = "rtsp/1.0";    private static final String TRACK_LINE = "a=control:trackID=";    private static final String TRANSPORT_DATA =      "TRANSPORT: UDP;unicast;client_port=8080-8081";    private static final String RTSP_OK = "RTSP/1.0 200 OK";    // base constructor, takes the media address, input and output streams    public RTSPProtocolHandler(        String address, InputStream is, OutputStream Os) {        this.address = address;        this.is = is;        this.os = Os;    }    // creates, sends and parses a DESCRIBE client request    public void doDescribe() throws IOException {        // if already described, return        if(described) return;        // create the base command        String baseCommand = getBaseCommand("DESCRIBE " + address);        // execute it and read the response        String response = doCommand(baseCommand);        // the response will contain track information, amongst other things        parseTrackInformation(response);        // set flag        described = true;    }    // creates, sends and parses a SETUP client request    public void doSetup() throws IOException {        // if not described        if(!described) throw new IOException("Not Described!");        // create the base command for the first SETUP track        String baseCommand =          getBaseCommand(                "SETUP " + address + "/trackID=" + tracks.elementAt(0));        // add the static transport data        baseCommand += CRLF + TRANSPORT_DATA;        // read response        String response = doCommand(baseCommand);        // parse it for session information        parseSessionInfo(response);        // if session information cannot be parsed, it is an error        if(sessionId == null)          throw new IOException("Could not find session info");        // now, send SETUP commands for each of the tracks        int cntOfTracks = tracks.size();        for(int i = 1; i < cntOfTracks; i++) {            baseCommand =                getBaseCommand(                    "SETUP " + address + "/trackID=" + tracks.elementAt(i));            baseCommand += CRLF + "Session: " + sessionId + CRLF + TRANSPORT_DATA;            doCommand(baseCommand);        }        // this is now setup        setup = true;    }    // issues a PLAY command    public void doPlay() throws IOException {        // must be first setup        if(!setup) throw new IOException("Not Setup!");        // create base command        String baseCommand = getBaseCommand("PLAY " + address);        // add session information        baseCommand += CRLF + "Session: " + sessionId;        // execute it        doCommand(baseCommand);        // set flags        playing = true;        stopped = false;    }    // issues a PAUSE command    public void doPause() throws IOException {        // if it is not playing, do nothing        if(!playing) return;        // create base command        String baseCommand = getBaseCommand("PAUSE " + address);        // add session information        baseCommand += CRLF + "Session: " + sessionId;        // execute it        doCommand(baseCommand);        // set flags        stopped = true;        playing = false;    }    // issues a TEARDOWN command    public void doTeardown() throws IOException {        // if not setup, nothing to teardown        if(!setup) return;        // create base command        String baseCommand = getBaseCommand("TEARDOWN " + address);        // add session information        baseCommand += CRLF + "Session: " + sessionId;        // execute it        doCommand(baseCommand);        // set flags        described = setup = playing = false;        stopped = true;    }    // this method is a convenience method to put a RTSP command together    private String getBaseCommand(String command) {        return(            command +            " " +            VERSION + // version            CRLF +            "CSeq: " + (CSeq++) // incrementing sequence        );    }    // executes a command and receives response from server    private String doCommand(String fullCommand) throws IOException {        // to read the response from the server        byte[] buffer = new byte[2048];        // debug        System.err.println(" ====== CLIENT REQUEST ====== ");        System.err.println(fullCommand + CRLF + CRLF);        System.err.println(" ============================ ");        // send a command        os.write((fullCommand + CRLF + CRLF).getBytes());        // read response        int length = is.read(buffer);        String response = new String(buffer, 0, length);        // empty the buffer        buffer = null;        // if the response doesn't start with an all clear        if(!response.startsWith(RTSP_OK))          throw new IOException("Server returned invalid code: " + response);        // debug        System.err.println(" ====== SERVER RESPONSE ====== ");        System.err.println(response.trim());        System.err.println(" =============================");        return response;    }    // convenience method to parse a server response to DESCRIBE command    // for track information    private void parseTrackInformation(String response) {        String localRef = response;        String trackId = "";        int index = localRef.indexOf(TRACK_LINE);        // iterate through the response to find all instances of the        // TRACK_LINE, which indicates all the tracks. Add all the        // track id's to the tracks vector        while(index != -1) {            int baseIdx = index + TRACK_LINE.length();            trackId = localRef.substring(baseIdx, baseIdx + 1);            localRef = localRef.substring(baseIdx + 1, localRef.length());            index = localRef.indexOf(TRACK_LINE);            tracks.addElement(trackId);        }    }    // find out the session information from the first SETUP command    private void parseSessionInfo(String response) {        sessionId =          response.substring(                response.indexOf("Session: ") + "Session: ".length(),                response.indexOf("Date:")).trim();    }}

Back to RTPSourceStream and StreamingDataSource

With the protocol handler in place, let's revisit the RTPSourceStream and
StreamingDataSource classes from earlier, where they contained only place-holder
methods. The StreamingDataSource is simple to code:

import java.io.IOException;import javax.microedition.media.Control;import javax.microedition.media.protocol.DataSource;import javax.microedition.media.protocol.SourceStream;public class StreamingDataSource extends DataSource {  // the full URL like locator to the destination  private String locator;  // the internal streams that connect to the source  // in this case, there is only one  private SourceStream[] streams;  // is this connected to its source?  private Boolean connected = false;  public StreamingDataSource(String locator) {      super(locator);      setLocator(locator);  }  public void setLocator(String locator) { this.locator = locator; }  public String getLocator() { return locator; }  public void connect() throws IOException {    // if already connected, return    if (connected) return;    // if locator is null, then can't actually connect    if (locator == null)      throw new IOException("locator is null");    // now populate the sourcestream array    streams = new RTPSourceStream[1];    // with a new RTPSourceStream    streams[0] = new RTPSourceStream(locator);    // set flag    connected = true;    }  public void disconnect() {    // if there are any streams    if (streams != null) {      // close the individual stream        try {          ((RTPSourceStream)streams[0]).close();        } catch(IOException ioex) {} // silent    }    // and set the flag    connected = false;  }  public void start() throws IOException {    if(!connected) return;    // start the underlying stream    ((RTPSourceStream)streams[0]).start();  }  public void stop() throws IOException {    if(!connected) return;    // stop the underlying stream    ((RTPSourceStream)streams[0])Close();  }  public String getContentType() {    // for the purposes of this article, it is only video/mpeg    return "video/mpeg";  }  public Control[] getControls() { return new Control[0]; }  public Control getControl(String controlType) { return null; }  public SourceStream[] getStreams() {    return streams; }}

The main work takes place in the connect() method. It creates a new
RTPSourceStream with the requested address. Notice that the
getContentType() method returnsvideo/mpeg as the default content type,
but change it to the supported content type for your system. Of course, this should not be hard-coded; it should be based on the actual support for different media types.

The next listing shows the complete RTPSourceStream class, which, along with
RTSPProtocolHandler, does the bulk of work in connecting getting the RTP packets of the
server:

import java.io.IOException;import java.io.InputStream;import java.io.OutputStream;import javax.microedition.io.Datagram;import javax.microedition.io.Connector;import javax.microedition.media.Control;import javax.microedition.io.SocketConnection;import javax.microedition.io.DatagramConnection;import javax.microedition.media.protocol.SourceStream;import javax.microedition.media.protocol.ContentDescriptor;public class RTPSourceStream implements SourceStream {    private RTSPProtocolHandler handler;    private InputStream is;    private OutputStream Os;    private DatagramConnection socket;    public RTPSourceStream(String address) throws IOException {        // create the protocol handler and set it up so that the        // application is ready to read data        // create a socketconnection to the remote host        // (in this case I have set it up so that its localhost, you can        // change it to wherever your server resides)        SocketConnection sc =          (SocketConnection)Connector.open("socket://localhost:554");        // open the input and output streams        is = sc.openInputStream();        Os = sc.openOutputStream();        // and initialize the handler        handler = new RTSPProtocolHandler(address, is, Os);        // send the basic signals to get it ready        handler.doDescribe();        handler.doSetup();    }    public void start() throws IOException {      // open a local socket on port 8080 to read data to      socket = (DatagramConnection)Connector.open("datagram://:8080");      // and send the PLAY command      handler.doPlay();    }    public void close() throws IOException {        if(handler != null) handler.doTeardown();        is.close();        os.close();    }    public int read(byte[] buffer, int offset, int length)      throws IOException {      // create a byte array which will be used to read the datagram      byte[] fullPkt = new byte[length];      // the new Datagram      Datagram packet = socket.newDatagram(fullPkt, length);      // receive it      socket.receive(packet);      // extract the actual RTP Packet's media data in the requested buffer      RTPPacket rtpPacket = getRTPPacket(packet, packet.getData());      buffer = rtpPacket.getData();      // debug      System.err.println(rtpPacket + " with media length: " + buffer.length);      // and return its length      return buffer.length;    }    // extracts the RTP packet from each datagram packet received    private RTPPacket getRTPPacket(Datagram packet, byte[] buf) {      // SSRC      long SSRC = 0;        // the payload type        byte PT = 0;      // the time stamp        int timeStamp = 0;        // the sequence number of this packet        short seqNo = 0;        // see http://www.networksorcery.com/enp/protocol/rtp.htm        // for detailed description of the packet and its data        PT =          (byte)((buf[1] & 0xff) & 0x7f);        seqNo =          (short)((buf[2] << 8) | ( buf[3] & 0xff));        timeStamp =          (((buf[4] & 0xff) << 24) | ((buf[5] & 0xff) << 16) |            ((buf[6] & 0xff) << 8) | (buf[7] & 0xff)) ;        SSRC =          (((buf[8] & 0xff) << 24) | ((buf[9] & 0xff) << 16) |            ((buf[10] & 0xff) << 8) | (buf[11] & 0xff));        // create an RTPPacket based on these values        RTPPacket rtpPkt = new RTPPacket();        // the sequence number        rtpPkt.setSequenceNumber(seqNo);        // the timestamp        rtpPkt.setTimeStamp(timeStamp);        // the SSRC        rtpPkt.setSSRC(SSRC);        // the payload type        rtpPkt.setPayloadType(PT);        // the actual payload (the media data) is after the 12 byte header        // which is constant        byte payload[] = new byte [packet.getLength() - 12];        for(int i=0; i < payload.length; i++) payload [i] = buf[i+12];        // set the payload on the RTP Packet        rtpPkt.setData(payload);        // and return the payload        return rtpPkt;    }    public long seek(long where) throws IOException {     throw new IOException("cannot seek");    }    public long tell() { return -1; }    public int getSeekType() { return NOT_SEEKABLE;    }    public Control[] getControls() { return null; }    public Control getControl(String controlType) { return null; }    public long getContentLength() { return -1;    }    public int getTransferSize() { return -1;    }    public ContentDescriptor getContentDescriptor() {        return new ContentDescriptor("audio/rtp");    }}

The constructor for the RTPSourceStream creates aSocketConnection to
the remote server (hard-coded to the local server and port here, but you can change this to accept
any server or port). It then opens the input and output streams, which it uses to create the
RTSPProtocolHandler. Finally, using this handler, it sends theDESCRIBE
and SETUP commands to the remote server to get the server ready to send the packets. The actual
delivery doesn't start until the start() method is called by the
StreamingDataSource, which opens up a local port (hard-coded to8081 in
this case) for receiving the packets and sends the PLAY command to start receiving
these packets. The actual reading of the packets is done in the read() method, which
receives the individual packets, strips them to create the RTPPacket instances (with
the getRTPPacket() method), and returns the media data in the buffer supplied while
calling the read() method.

A MIDlet to see if it works

With all the classes in place, let's write a simple MIDlet to first create a Player
instance that will use the StreamingDataSource to connect to the server and then get
media packets from it. The Player interface is defined by the MMAPI and allows you to
control the playback (or recording) of media. Instances of this interface are created by using the
Manager class from the MMAPIjavax.microedition.media package (see the <a
href="http://today.java.net/pub/a/today/2005/09/27/j2me4.html">MMAPI tutorial). The following
shows this rudimentary MIDlet:

import javax.microedition.media.Player;import javax.microedition.midlet.MIDlet;import javax.microedition.media.Manager;public class StreamingMIDlet extends MIDlet {  public void startApp() {    try {      // create Player instance, realize it and then try to start it      Player player =        Manager.createPlayer(          new StreamingDataSource(            "rtsp://localhost:554/sample_100kbit.mp4"));      player.realize();      player.start();    } catch(Exception e) {            e.printStackTrace();    }  }  public void pauseApp() {}  public void destroyApp(boolean unconditional) {}}

So what should happen when you run this MIDlet in the Wireless toolkit? I have on purpose left
out any code to display the resulting video on screen. When I run it in the toolkit, I know that I
am receiving the packets because I see the debug statements as shown in Figure 2.

Figure 2. Running StreamingMIDlet output

The RTP packets as sent by the server are being received. The StreamingDataSource
along with the RTSPProtocolHandler andRTPSourceStream are doing their job
of making the streaming server send these packets. This is confirmed by looking at the streaming
server's admin console as shown in Figure 3.

<a href="http://today.java.net/images/2006/08/experiments-figure3.gif"
onclick="window.open('/images/2006/08/experiments-figure3.gif','fullsize','toolbar=no,width=635,
height=239,status=no,location=no,scrollbars=yes,resizable=yes,menubar=yes');return false"><img
src="/images/2006/08/experiments-figure3-sm.gif" vspace="4" alt="Figure 3 - Darwin's admin console
shows that the file is being streamed" width="450" height="169" border="0" />
Figure
3. Darwin's admin console shows that the file is being streamed (click for full-size
image).

Unfortunately, the player constructed by the Wireless toolkit is trying to read the entire content
at one go. Even if I were to make a StreamingVideoControl, it will not display the
video until it has read the whole file, therefore defeating the purpose of the streaming aspect of this
whole experiment. So what needs to be done to achieve the full streaming experience?

Ideally, MMAPI should provide the means for developers to register the choice of
Player for the playback of certain media. This is easily achieved by providing a new
method in the Manager class for registering (or overriding) MIME types or protocols
with developer-made Player instances. For example, let's say I create a Player instance
that reads streaming data called StreamingMPEGPlayer. With the Manager class, I should be able to
say Manager.registerPlayer("video/mpeg",StreamingMPEGPlayer.class) or
Manager.registerPlayer("rtsp",StreamingMPEGPlayer.class). MMAPI should then simply
load this developer-made Player instance and use this as the means to read data from
the developer-made datasource.

In a nutshell, you need to be able to create an independent media player and register it as the
choice of instance for playing the desired content. Unfortunately, this is not possible with the
current MMAPI implementation, and this is the data consumption conundrum that I had talked about
earlier.

Of course, if you can test this code in a toolkit that does not need to read the complete data
before displaying it (or for audio files, playing them), then you have achieved the aim of streaming
data using the existing MMAPI implementation.

This experiment should prove that you can stream data with the current MMAPI implementation, but
you may not be able to manipulate it in a useful manner until you have better control over the
Manager and Player instances. I look forward to your comments and
experiments using this code.

Resources

• Sample code for this article
• Sun's MMAPI page
• Simple RTSP Client Steps required to create an RTSP client
• RTSP RFC andRTP RFC
• Apple's Darwin Stream Server information page
• Mobile Media API (MMAPI) Book
• Mobile Media API (MMAPI) Tutorial