Experiments in Streaming Content in Java ME

Experiments in Streaming Content in Java ME

by Vikram Goyal
08/22/2006

• Contents
• Background to the streaming problem
  • What is streaming?
  • HTTP and RTSP
  • Making sense of RTSP, RTP and RTCP
• Set up a streaming server
• Model an RTP packet
• Create a custom DataSource
• Creating an RTSP Protocol Handler
• Back to RTPSourceStream and StreamingDataSource
• A MIDlet to see if it works
• Resources

Since my book on Mobile Media API (MMAPI), Pro Java ME MMAPI: Mobile Media API for Java Micro Edition, was published in May, I have been inundated with requests to help readers with streaming content via MMAPI for Java-enabled mobile devices. This topic was an important omission from the book, but one that was simply not feasible to include because of the lack of support for it within various MMAPI implementations. In this article, I will show you the results of experiments I have conducted since the publication of the book to stream content via MMAPI using a custom datasource.

DISCLAIMER: Before I commence, I would like to point out that even though I was able to stream data from a streaming server and receive it successfully in a MIDlet using a custom datasource, I wasn't able to utilize this data in any meaningful manner because of limitations in the way this data is read by the MMAPI implementation at my disposal. You may have more success if you have access to a MMAPI implementation that doesn't read its data fully. Even if you don't, this article still provides a good study of the issues involved in streaming media data. At the very least, it shows you how to create and utilize your own custom datasource.

For a background on Java ME please see my previous tutorial series on getting started. For an introduction to MMAPI, tutorial 4 is a good start, or you can always buy the book.

Background to the streaming problem

MMAPI is a format- and protocol-agnostic API, which means that the API doesn't dictate mandatory support from device manufacturers for any particular format or protocol. One of the protocols that is widely requested by application developers is the Real Time Streaming Protocol (RTSP) and the associated Real-time Transport Protocol (RTP) for streaming audio/video content. The advantage of streaming content is that it provides a fast turnaround time for the user, control over the content distribution to the distributor, and an overall richer user experience.

However, hardly any manufacturer supports this protocol through Java ME. Some new phones provide support for RTSP, but that support is only on a smattering of devices. A majority of devices still do not support this protocol, therefore limiting useful application development in the streaming media department. A majority of questions in the MMAPI forums of various device manufacturers revolve around this very issue, that is, how to provide streaming data when RTSP is not supported. This article aims to point you in the right direction. I'll start by cutting through the clutter to try to provide an understanding of what streaming means.

What is streaming?

Streaming is the process of transferring data via a channel to its destination, where it is decoded and consumed via the user or device in real time, that is, as the data is being delivered. It differs from non-streaming processes because it doesn't require the data to be fully downloaded before it can be seen or used. Streaming is not the property of the data that is being delivered, but is an attribute of the distribution channel. This means, technically, that most media can be streamed.

HTTP and RTSP

HTTP and RTSP are application-level protocols that allow remote retrieval of data. So why can't you use HTTP for streaming media content? The truth is, you can. When you click on a Web page link to play an audio file, in most cases the media data is streamed to your machine. However, streaming content over HTTP is inherently inefficient. This is because HTTP is based on the Transmission Control Protocol (TCP), which makes sure that media packets are delivered to their destination reliably without worrying about when they are delivered. On the other hand, RTSP can be based on both User Datagram Protocol (UDP), which is a connectionless protocol ensuring faster delivery over reliability, and on TCP. Besides, RTSP has control mechanisms built in that allow random access to the media data, allowing you to seek, pause, and play.

Making sense of RTSP, RTP, and RTCP

There is a lot of confusion among newcomers over the acronyms RTSP, RTP, and RTCP. All three represent different protocols related to streaming of media content. An RTSP session initiates both Real-time Transport Protocol (RTP) and RTP Control Protocol (RTCP) sessions. RTSP is only the control protocol, a bit like a remote control for a DVD player, in that it allows you to start, stop, resume, and seek data remotely. The actual data delivery is done via RTP, and RTCP is a partner protocol to RTP providing feedback to both the sender and receiver on the quality of media data that is being transferred.

With this basic introduction about RTSP and streaming out of the way, let's set up our own streaming server to conduct some experiments. You can read more about RTSP, RTP, and RTCP at http://www.rtsp.org.

Set up a streaming server

To conduct experiments for the purposes of this article, you will need access to a specialty streaming server that can create RTSP streams for media objects. One such server is the Darwin Streaming Server, which is an open-source streaming server based on the same source code as Apple's commercial QuickTime streaming server. Implementations of this free server are available for Mac OS, Linux, and Windows. Download the version that is suitable for your OS and run the installer. You can also choose to download the source code and build it in your environment. I have run the examples in this article on a Windows XP machine, and the server is installed in C:/Program Files/Darwin Streaming Server .

For the purposes of this article, you will also need to have Perl installed on your computer, to administer the Darwin server. For Windows, you can download ActivePerl.

As part of the installation, you will be asked to provide an administrator username and password, but make sure that you run the administration server after the installation (by running the streamingadminserver.pl file). This starts an administration server on port 1220 with which you can monitor the current activity within the streaming server. More importantly, you will need to supply a username/password combination the first time you log into the administrative console (by navigating to http://localhost:1220 in your browser) for running the movie and MP3 broadcast service. It is important to set this (even though you never really need to supply this username/password combination anywhere when running the examples in this article).

Note: On Windows, if you download the latest version of ActivePerl, streamingadminserver.pl is likely to fail with the following error:

ActivePerl 5.8.0 or higher is required 
  in order to run the Darwin Streaming Server web-based administration. Please 
  download it from http://www.activeperl.com/ and install it.

This is because of an incorrect configuration check in this script, and you can easily fix it by commenting out lines 33 and 34 (put a # in front of these lines).

The streaming server starts on port 554 and comes with a few sample movie files, ready for streaming in the installation folder under the Movies directory. The Darwin server can stream MPEG-4, 3GPP, and QuickTime movie files natively. This means that these files don't need to be "hinted" in order to be streamed. Hinting is a process by which media files are prepared with track information for streaming using the professional version of QuickTime. For the purposes of this article, I will work with natively streamable files like 3GPP and MPEG-4 only.

To test that your streaming server is working correctly, use the QuickTime player to launch a file via RTSP. For example, if you can open the URL rtsp://localhost:554/sample_50kbit.3gp correctly in the Quicktime player and view the file, pause it, stop it, and seek it, then your streaming server is working correctly.

Model an RTP packet

As I said earlier, RTP is the actual delivery protocol for streaming data. Each streaming session involves the streaming server sending RTP packets to its destination based on the client request (requests that are delivered via the RTSP protocol). A full knowledge of the RTP RFC is not required for the purposes of this article, so the following base class will model an RTP packet to its best possible approximation.

Note: I have used the Java ME Wireless Toolkit 2.3 (beta) to create and run the examples in this article. You can start by creating a project called "StreamingData" (or whatever you prefer) in this toolkit to place your code in. The J2ME tutorial part 1 gives more details on the process of creating projects in this toolkit.

public class RTPPacket {

  // used to identify separate streams that may contribute to this packet
  private long SSRC;

  // incrementing identifier for each packet that is sent
  private long sequenceNumber;

  // used to place this packet in the correct timing order
  // that is, where this packet fits in time based media
  private long timeStamp;

  // the type of the media data, or the payload type
  private long payloadType;

  // the actual media data, also called the payload
  private byte data[];

  // the get and set methods
  public long getSSRC() { return this.SSRC; }
  public void setSSRC(long SSRC) { this.SSRC = SSRC; }

  public long getSequenceNumber() { return this.sequenceNumber; }
  public void setSequenceNumber(long sequenceNumber)
    { this.sequenceNumber = sequenceNumber; }

  public long getTimeStamp() { return this.timeStamp; }
  public void setTimeStamp(long timeStamp) { this.timeStamp = timeStamp; }

  public long getPayloadType() { return this.payloadType; }
  public void setPayloadType(long payloadType)
    { this.payloadType = payloadType; }

  public byte[] getData() { return this.data; }
  public void setData(byte[] data) { this.data = data; }

  public String toString() {
    return
      "RTPPacket " + sequenceNumber +
      ": [" +
      " ssrc=0x" + SSRC +
      ", timestamp=" + timeStamp +
      ", payload type=" + payloadType +
      " ]";

  }

}

The comments within the code should offer you some idea about the various features of an RTP packet. Since you won't be building a complete RTP client and will be running this code within the confines of this example, the main feature of the above class is the data, or the payload contained within such a packet. Note that an RTP packet contains other information as well, which is not modeled by this class.

Create a custom DataSource

A DataSource is a MMAPI abstract class, implementations of which encapsulate the task of media data location and retrieval. Device manufacturers provide their own implementations in the Java ME toolkit for most sources. Developers don't need to create their own custom datasources because the task of locating data over file or network is rudimentary and fulfilled by the device manufacturer's implementation. However, in cases where the developer needs to do data retrieval from a custom source, a custom datasource is the answer, and media data fetched from a streaming server is a perfect example.

Data retrieval is one thing, while data consumption is another. Since MMAPI doesn't allow you to create custom media players, will a custom datasource suffice in this example? Let's proceed further with the creation of the custom datasource before I answer that question. The following listing shows the starting of the custom datasource class that I will use for talking to the streaming server:

import javax.microedition.media.Control;
import javax.microedition.media.protocol.DataSource;
import javax.microedition.media.protocol.SourceStream;

public class StreamingDataSource extends DataSource {

  // the full URL like locator to the destination
  private String locator;

  // the internal stream that connects to the source
  private SourceStream[] streams;

  public StreamingDataSource(String locator) {
    super(locator);
    setLocator(locator);
  }

  public void setLocator(String locator) { this.locator = locator; }

  public String getLocator() { return locator; }

  public void connect() {}

  public void stop() {}

  public void start() {}

  public void disconnect() {}

  public String getContentType() { return ""; }

  public Control[] getControls() { return null; }

  public Control getControl(String controlType) { return null;  }

  public SourceStream[] getStreams() {    return streams;  }

}

This class contains only placeholder methods at the moment. Internally, each datasource uses a SourceStream implementation to read individual streams of data from; therefore, let's create a simple SourceStream implementation for reading RTP packets:

import java.io.IOException;
import javax.microedition.media.Control;
import javax.microedition.media.protocol.SourceStream;
import javax.microedition.media.protocol.ContentDescriptor;

public class RTPSourceStream implements SourceStream {

  public RTPSourceStream(String address) throws IOException { }

  public void close() { }

  public int read(byte[] buffer, int offset, int length)
   throws IOException {

   return 0;
  }

  public long seek(long where) throws IOException {
   throw new IOException("cannot seek");
  }

  public long tell() { return -1; }

  public int getSeekType() { return NOT_SEEKABLE;    }

  public Control[] getControls() { return null; }

  public Control getControl(String controlType) { return null; }

  public long getContentLength() { return -1;    }

  public int getTransferSize() { return -1;    }

  public ContentDescriptor getContentDescriptor() {
      return new ContentDescriptor("audio/rtp");
  }
}

As with the previous listing, this class only contains placeholder methods for the moment. However, all listings so far should compile and preverify successfully.

Creating an RTSP Protocol Handler

Recall that RTSP is the actual protocol over which streaming commands are initiated, through which the RTP packets are received. The RTSP protocol is like a command initiator, a bit like HTTP. For a really good explanation of a typical RTSP session, please see these specifications for a simple RTSP client. For the purposes of this article, I am going to oversimplify the protocol implementation. Figure 1 shows the typical RTSP session between a client and a streaming server.

Figure 1. A typical RTSP session between a RTSP client and a streaming server (click for full-size image).

In a nutshell, an RTSP client initiates a session by sending a DESCRIBE request to the streaming server which means that the client wants more information about a media file. An example DESCRIBE request may look like this:

DESCRIBE rtsp://localhost:554/media.3gp rtsp/1.0
CSeq: 1

The URL for the media file is followed by the RTSP version that the client is following, and a carriage return/line feed (CRLF). The next line contains the sequence number of this request and increments for each subsequent request sent to the server. The command is terminated by a single line on its own (as are all RTSP commands).

All client commands that are successful receive a response that starts with RTSP/1.0 200 OK. For the DESCRIBE request, the server responds with several parameters, and if the file is present and streamable, this response contains any information for any tracks in special control strings that start with a a=control:trackID= String. The trackID is important and is used to create the next requests to the server.

Once described, the media file's separate tracks are set up for streaming using the SETUP command, and these commands should indicate the transport properties for the subsequent RTP packets. This is shown here:

SETUP rtsp://localhost:554/media.3gp/trackID=3 rtsp/1.0
CSeq: 2
TRANSPORT: UDP;unicast;client_port=8080-8081

The previous command indicates to the server to set up to stream trackID 3 of the media.3gp file, to send the packets via UDP, and to send them to port 8080 on the client (8081 is for RTCP commands). The response to the first SETUP command (if it is okay) will contain the session information for subsequent commands and must be included as shown here:

SETUP rtsp://localhost:554/media.3gp/trackID=3 rtsp/1.0
CSeq: 3
Session: 556372992204
TRANSPORT: UDP;unicast;client_port=8080-8081

An OK response from the server indicates that you can send the PLAY command, which will make the server start sending the RTP packets:

PLAY rtsp://localhost:554/media.3gp rtsp/1.0
CSeq: 3
Session: 556372992204

Notice that the PLAY command is issued only on the main media file, and not on any individual tracks. The same is true for the PAUSE and TEARDOWN commands, which are identical to the PLAY command, except for the command itself.

The following listing contains the RTSPProtocolHandler class. The comments in the code and the brief information so far should help with understanding how this protocol handler works:

import java.util.Vector;
import java.io.InputStream;
import java.io.IOException;
import java.io.OutputStream;

public class RTSPProtocolHandler {

    // the address of the media file as an rtsp://... String
    private String address;

    // the inputstream to receive response from the server
    private InputStream is;

    // the outputstream to write to the server
    private OutputStream os;

    // the incrementing sequence number for each request
    // sent by the client
    private static int CSeq = 1;

    // the session id sent by the server after an initial setup
    private String sessionId;

    // the number of tracks in a media file
    private Vector tracks = new Vector(2);

    // flags to indicate the status of a session
    private boolean described, setup, playing;
    private Boolean stopped = true;

    // constants
    private static final String CRLF = "/r/n";
    private static final String VERSION = "rtsp/1.0";
    private static final String TRACK_LINE = "a=control:trackID=";
    private static final String TRANSPORT_DATA =
      "TRANSPORT: UDP;unicast;client_port=8080-8081";
    private static final String RTSP_OK = "RTSP/1.0 200 OK";

    // base constructor, takes the media address, input and output streams
    public RTSPProtocolHandler(
        String address, InputStream is, OutputStream Os) {

        this.address = address;
        this.is = is;
        this.os = Os;
    }

    // creates, sends and parses a DESCRIBE client request
    public void doDescribe() throws IOException {

        // if already described, return
        if(described) return;

        // create the base command
        String baseCommand = getBaseCommand("DESCRIBE " + address);

        // execute it and read the response
        String response = doCommand(baseCommand);

        // the response will contain track information, amongst other things
        parseTrackInformation(response);

        // set flag
        described = true;
    }

    // creates, sends and parses a SETUP client request
    public void doSetup() throws IOException {

        // if not described
        if(!described) throw new IOException("Not Described!");

        // create the base command for the first SETUP track
        String baseCommand =
          getBaseCommand(
                "SETUP " + address + "/trackID=" + tracks.elementAt(0));

        // add the static transport data
        baseCommand += CRLF + TRANSPORT_DATA;

        // read response
        String response = doCommand(baseCommand);

        // parse it for session information
        parseSessionInfo(response);

        // if session information cannot be parsed, it is an error
        if(sessionId == null)
          throw new IOException("Could not find session info");

        // now, send SETUP commands for each of the tracks
        int cntOfTracks = tracks.size();
        for(int i = 1; i < cntOfTracks; i++) {
            baseCommand =
                getBaseCommand(
                    "SETUP " + address + "/trackID=" + tracks.elementAt(i));
            baseCommand += CRLF + "Session: " + sessionId + CRLF + TRANSPORT_DATA;
            doCommand(baseCommand);
        }

        // this is now setup
        setup = true;
    }

    // issues a PLAY command
    public void doPlay() throws IOException {

        // must be first setup
        if(!setup) throw new IOException("Not Setup!");

        // create base command
        String baseCommand = getBaseCommand("PLAY " + address);

        // add session information
        baseCommand += CRLF + "Session: " + sessionId;

        // execute it
        doCommand(baseCommand);

        // set flags
        playing = true;
        stopped = false;
    }

    // issues a PAUSE command
    public void doPause() throws IOException {

        // if it is not playing, do nothing
        if(!playing) return;

        // create base command
        String baseCommand = getBaseCommand("PAUSE " + address);

        // add session information
        baseCommand += CRLF + "Session: " + sessionId;

        // execute it
        doCommand(baseCommand);

        // set flags
        stopped = true;
        playing = false;
    }

    // issues a TEARDOWN command
    public void doTeardown() throws IOException {

        // if not setup, nothing to teardown
        if(!setup) return;

        // create base command
        String baseCommand = getBaseCommand("TEARDOWN " + address);

        // add session information
        baseCommand += CRLF + "Session: " + sessionId;

        // execute it
        doCommand(baseCommand);

        // set flags
        described = setup = playing = false;
        stopped = true;
    }

    // this method is a convenience method to put a RTSP command together
    private String getBaseCommand(String command) {

        return(
            command +
            " " +
            VERSION + // version
            CRLF +
            "CSeq: " + (CSeq++) // incrementing sequence
        );
    }

    // executes a command and receives response from server
    private String doCommand(String fullCommand) throws IOException {

        // to read the response from the server
        byte[] buffer = new byte[2048];

        // debug
        System.err.println(" ====== CLIENT REQUEST ====== ");
        System.err.println(fullCommand + CRLF + CRLF);
        System.err.println(" ============================ ");

        // send a command
        os.write((fullCommand + CRLF + CRLF).getBytes());

        // read response
        int length = is.read(buffer);

        String response = new String(buffer, 0, length);

        // empty the buffer
        buffer = null;

        // if the response doesn't start with an all clear
        if(!response.startsWith(RTSP_OK))
          throw new IOException("Server returned invalid code: " + response);

        // debug
        System.err.println(" ====== SERVER RESPONSE ====== ");
        System.err.println(response.trim());
        System.err.println(" =============================");

        return response;
    }

    // convenience method to parse a server response to DESCRIBE command
    // for track information
    private void parseTrackInformation(String response) {

        String localRef = response;
        String trackId = "";
        int index = localRef.indexOf(TRACK_LINE);

        // iterate through the response to find all instances of the
        // TRACK_LINE, which indicates all the tracks. Add all the
        // track id's to the tracks vector
        while(index != -1) {
            int baseIdx = index + TRACK_LINE.length();
            trackId = localRef.substring(baseIdx, baseIdx + 1);
            localRef = localRef.substring(baseIdx + 1, localRef.length());
            index = localRef.indexOf(TRACK_LINE);
            tracks.addElement(trackId);
        }

    }

    // find out the session information from the first SETUP command
    private void parseSessionInfo(String response) {

        sessionId =
          response.substring(
                response.indexOf("Session: ") + "Session: ".length(),
                response.indexOf("Date:")).trim();

    }

Back to RTPSourceStream and StreamingDataSource

With the protocol handler in place, let's revisit the RTPSourceStream and StreamingDataSource classes from earlier, where they contained only place-holder methods. The StreamingDataSource is simple to code:

import java.io.IOException; import javax.microedition.media.Control; import javax.microedition.media.protocol.DataSource; import javax.microedition.media.protocol.SourceStream; public class StreamingDataSource extends DataSource { // the full URL like locator to the destination private String locator; // the internal streams that connect to the source // in this case, there is only one private SourceStream[] streams; // is this connected to its source? private Boolean connected = false; public StreamingDataSource(String locator) { super(locator); setLocator(locator); } public void setLocator(String locator) { this.locator = locator; } public String getLocator() { return locator; } public void connect() throws IOException { // if already connected, return if (connected) return; // if locator is null, then can't actually connect if (locator == null) throw new IOException("locator is null"); // now populate the sourcestream array streams = new RTPSourceStream[1]; // with a new RTPSourceStream streams[0] = new RTPSourceStream(locator); // set flag connected = true; } public void disconnect() { // if there are any streams if (streams != null) { // close the individual stream try { ((RTPSourceStream)streams[0]).close(); } catch(IOException ioex) {} // silent } // and set the flag connected = false; } public void start() throws IOException { if(!connected) return; // start the underlying stream ((RTPSourceStream)streams[0]).start(); } public void stop() throws IOException { if(!connected) return; // stop the underlying stream ((RTPSourceStream)streams[0])Close(); } public String getContentType() { // for the purposes of this article, it is only video/mpeg return "video/mpeg"; } public Control[] getControls() { return new Control[0]; } public Control getControl(String controlType) { return null; } public SourceStream[] getStreams() { return streams; } }

The main work takes place in the connect() method. It creates a new RTPSourceStream with the requested address. Notice that the getContentType() method returns video/mpeg as the default content type, but change it to the supported content type for your system. Of course, this should not be hard-coded; it should be based on the actual support for different media types.

The next listing shows the complete RTPSourceStream class, which, along with RTSPProtocolHandler, does the bulk of work in connecting getting the RTP packets of the server:

import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import javax.microedition.io.Datagram; import javax.microedition.io.Connector; import javax.microedition.media.Control; import javax.microedition.io.SocketConnection; import javax.microedition.io.DatagramConnection; import javax.microedition.media.protocol.SourceStream; import javax.microedition.media.protocol.ContentDescriptor; public class RTPSourceStream implements SourceStream { private RTSPProtocolHandler handler; private InputStream is; private OutputStream Os; private DatagramConnection socket; public RTPSourceStream(String address) throws IOException { // create the protocol handler and set it up so that the // application is ready to read data // create a socketconnection to the remote host // (in this case I have set it up so that its localhost, you can // change it to wherever your server resides) SocketConnection sc = (SocketConnection)Connector.open("socket://localhost:554"); // open the input and output streams is = sc.openInputStream(); Os = sc.openOutputStream(); // and initialize the handler handler = new RTSPProtocolHandler(address, is, Os); // send the basic signals to get it ready handler.doDescribe(); handler.doSetup(); } public void start() throws IOException { // open a local socket on port 8080 to read data to socket = (DatagramConnection)Connector.open("datagram://:8080"); // and send the PLAY command handler.doPlay(); } public void close() throws IOException { if(handler != null) handler.doTeardown(); is.close(); os.close(); } public int read(byte[] buffer, int offset, int length) throws IOException { // create a byte array which will be used to read the datagram byte[] fullPkt = new byte[length]; // the new Datagram Datagram packet = socket.newDatagram(fullPkt, length); // receive it socket.receive(packet); // extract the actual RTP Packet's media data in the requested buffer RTPPacket rtpPacket = getRTPPacket(packet, packet.getData()); buffer = rtpPacket.getData(); // debug System.err.println(rtpPacket + " with media length: " + buffer.length); // and return its length return buffer.length; } // extracts the RTP packet from each datagram packet received private RTPPacket getRTPPacket(Datagram packet, byte[] buf) { // SSRC long SSRC = 0; // the payload type byte PT = 0; // the time stamp int timeStamp = 0; // the sequence number of this packet short seqNo = 0; // see http://www.networksorcery.com/enp/protocol/rtp.htm // for detailed description of the packet and its data PT = (byte)((buf[1] & 0xff) & 0x7f); seqNo = (short)((buf[2] << 8) | ( buf[3] & 0xff)); timeStamp = (((buf[4] & 0xff) << 24) | ((buf[5] & 0xff) << 16) | ((buf[6] & 0xff) << 8) | (buf[7] & 0xff)) ; SSRC = (((buf[8] & 0xff) << 24) | ((buf[9] & 0xff) << 16) | ((buf[10] & 0xff) << 8) | (buf[11] & 0xff)); // create an RTPPacket based on these values RTPPacket rtpPkt = new RTPPacket(); // the sequence number rtpPkt.setSequenceNumber(seqNo); // the timestamp rtpPkt.setTimeStamp(timeStamp); // the SSRC rtpPkt.setSSRC(SSRC); // the payload type rtpPkt.setPayloadType(PT); // the actual payload (the media data) is after the 12 byte header // which is constant byte payload[] = new byte [packet.getLength() - 12]; for(int i=0; i < payload.length; i++) payload [i] = buf[i+12]; // set the payload on the RTP Packet rtpPkt.setData(payload); // and return the payload return rtpPkt; } public long seek(long where) throws IOException { throw new IOException("cannot seek"); } public long tell() { return -1; } public int getSeekType() { return NOT_SEEKABLE; } public Control[] getControls() { return null; } public Control getControl(String controlType) { return null; } public long getContentLength() { return -1; } public int getTransferSize() { return -1; } public ContentDescriptor getContentDescriptor() { return new ContentDescriptor("audio/rtp"); } }

The constructor for the RTPSourceStream creates a SocketConnection to the remote server (hard-coded to the local server and port here, but you can change this to accept any server or port). It then opens the input and output streams, which it uses to create the RTSPProtocolHandler. Finally, using this handler, it sends the DESCRIBE and SETUP commands to the remote server to get the server ready to send the packets. The actual delivery doesn't start until the start() method is called by the StreamingDataSource, which opens up a local port (hard-coded to 8081 in this case) for receiving the packets and sends the PLAY command to start receiving these packets. The actual reading of the packets is done in the read() method, which receives the individual packets, strips them to create the RTPPacket instances (with the getRTPPacket() method), and returns the media data in the buffer supplied while calling the read() method.

A MIDlet to see if it works

With all the classes in place, let's write a simple MIDlet to first create a Player instance that will use the StreamingDataSource to connect to the server and then get media packets from it. The Player interface is defined by the MMAPI and allows you to control the playback (or recording) of media. Instances of this interface are created by using the Manager class from the MMAPI javax.microedition.media package (see the MMAPI tutorial). The following shows this rudimentary MIDlet:

import javax.microedition.media.Player; import javax.microedition.midlet.MIDlet; import javax.microedition.media.Manager; public class StreamingMIDlet extends MIDlet { public void startApp() { try { // create Player instance, realize it and then try to start it Player player = Manager.createPlayer( new StreamingDataSource( "rtsp://localhost:554/sample_100kbit.mp4")); player.realize(); player.start(); } catch(Exception e) { e.printStackTrace(); } } public void pauseApp() {} public void destroyApp(boolean unconditional) {} }

So what should happen when you run this MIDlet in the Wireless toolkit? I have on purpose left out any code to display the resulting video on screen. When I run it in the toolkit, I know that I am receiving the packets because I see the debug statements as shown in Figure 2.

Figure 2. Running StreamingMIDlet output

The RTP packets as sent by the server are being received. The StreamingDataSource along with the RTSPProtocolHandler and RTPSourceStream are doing their job of making the streaming server send these packets. This is confirmed by looking at the streaming server's admin console as shown in Figure 3.

Figure 3. Darwin's admin console shows that the file is being streamed (click for full-size image).

Unfortunately, the player constructed by the Wireless toolkit is trying to read the entire content at one go. Even if I were to make a StreamingVideoControl, it will not display the video until it has read the whole file, therefore defeating the purpose of the streaming aspect of this whole experiment. So what needs to be done to achieve the full streaming experience?

Ideally, MMAPI should provide the means for developers to register the choice of Player for the playback of certain media. This is easily achieved by providing a new method in the Manager class for registering (or overriding) MIME types or protocols with developer-made Player instances. For example, let's say I create a Player instance that reads streaming data called StreamingMPEGPlayer. With the Manager class, I should be able to say Manager.registerPlayer("video/mpeg", StreamingMPEGPlayer.class) or Manager.registerPlayer("rtsp", StreamingMPEGPlayer.class). MMAPI should then simply load this developer-made Player instance and use this as the means to read data from the developer-made datasource.

In a nutshell, you need to be able to create an independent media player and register it as the choice of instance for playing the desired content. Unfortunately, this is not possible with the current MMAPI implementation, and this is the data consumption conundrum that I had talked about earlier.

Of course, if you can test this code in a toolkit that does not need to read the complete data before displaying it (or for audio files, playing them), then you have achieved the aim of streaming data using the existing MMAPI implementation.

This experiment should prove that you can stream data with the current MMAPI implementation, but you may not be able to manipulate it in a useful manner until you have better control over the Manager and Player instances. I look forward to your comments and experiments using this code.

Resources

• Sample code for this article
• Sun's MMAPI page
• Simple RTSP Client Steps required to create an RTSP client
• RTSP RFC and RTP RFC
• Apple's Darwin Stream Server information page
  
• Mobile Media API (MMAPI) Book
• Mobile Media API (MMAPI) Tutorial

Vikram Goyal is the author of Pro Java ME MMAPI.

}