This project, High quality multimedia delivery over IP networks which was developed by myself, is a subset of the Synchronous Distance Education System developed by New Zealand Educational Software Centre. It has developed a multimedia delivery system to support bi-directional high quality video, audio, chat messages and controls signalsdelivery over network in real-time. This software is based on Apple Macintosh system.

To develop this system, there are four items have been considered: high quality video and audio, high data bit rate transmission, data loss and communication delay.

For the delivery of live educational presentations, the video should be clear enough to precisely illustrate the concepts of the teaching, say full screen display or 720 * 576 pixels resolution. Then it needs a large amount of data throughput over the network to deliver essential video and audio information. To guarantee this transmission, a reliable network and its protocol are needed. Video transmission requires a huge data rate in the network and data loss is inevitable. To reduce data loss to a minimum is the one of the aims of this project. Picture compression will be used in this system. As the picture compression and decompression need time to be accomplished, so communication delay is unavoidable. Just like eliminate data loss, reducing compression delay is another important optimisation for this system.

This system has been designed and implemented from scratch and has achieved following design goals:

This system delivers lectures over Ethernet to the students’ end with fully-animated pictures and voices. Similarly, it also sends fully-animated pictures of the students and voices back to the lecturer’s end simultaneously. There are two streams for each one-way communication over the LAN. One is an MPEG multiplexed system stream (video and audio data), the other is the control stream, which conveys control signals and whiteboard data. The number of streams is doubled in bi-directional communication (See Figure 4-1 ). The following descriptions are assumed to be for the bi-directional communication.

Figure 4-1 Streaming diagram

The lecturer's audio and video streams are encoded by the MPEG encoder card and a multiplexed MPEG system stream is generated, which mixes the audio and video signals together with the time marks. Then that stream is divided into small packets and sent over the LAN using the UDP protocol. At the other end (the students' end), the packets are received and reassembled into the original MPEG system stream, assuming no packet loss, or something closely approximating it if there is packet loss. Then the stream is decoded into separate audio and video streams by the MPEG decoder card. The audio and video signals are fed directly to the display monitor. This exact procedure is also applicable to the students' end. So there are two channels that support this type of communication.

The other two channels are used to exchange some essential control messages and allow remote configuration, such as adjustments to the display window's size and position. The control channel is also used as a whiteboard communication channel. Data can be directly sent to the network using TCP/IP protocol if its size is less than 64K. Otherwise it needs to be divided into small packets* . At the other end of the network, the control signal is received and corresponding activity is carried out. Figure 4-1 shows an example of the live communication between lecturer and students.

There are three components in this system. They are the Interface and control component that supports different windows, menus for settings and display of status, videos and chat messages, the Codec component that supports MPEG1 and MPEG2 encode and decode and the Network component that manages the network connections, transmissions and protocols (See Figure 4-2).

Figure 4-2 Project components

3.1 Interface and control component

This is the main control and interfacing component that supports human-computer communication and message display. It has seven modules:

• System controller: This is a main control module that sets up and controls the whole system. This module contains two group interfaces, Main menu and Control panel.

• Main menu: the main menu of the application that specifies the type of encoding (MPEG1/MPEG2), the bit rate of the encoded MPEG data stream, the video input source, etc.
• Control panel: this is a control pad that creates a new broadcast session, creates a client to receive an MPEG stream, records or stops recording an incoming stream, plays, pauses, stills, still steps or stops an MPEG video stored on the disk. It can also increase or decrease the video preview window size, set the remote window control on or off and show or hide the video preview window and chat window.

• Network connection: a dialogue window for the user to specify the remote host IP address and port number when it receives a remote MPEG stream. It also can set its own port number for this communication.
• Decode & incoming stream status: a message window that displays the incoming data bit rate, MPEG decoder card status, the IP addresses of both communicating ends and the communication time elapsed.
• Encode & outgoing stream status: a message window that displays the outgoing data bit rate, MPEG encoder card status, the IP addresses of both communicating ends and the communication time elapsed.
• Video displayer: a preview window that plays the received MPEG stream with synchronised sound. It also can adjust its window size and position when requested.
• Chat manager: support whiteboard communication through a text dialogue window.

Figure 4-3 Screen shot

The Codec component performs MPEG encoding and decoding functions. There are two modules in this component, the Encoder and the Decoder.

• Encoder: this sets the MPEG encoding parameters and controls the Wired Inc.'s encoder card to begin or stop capturing the video or to record the captured video. Also it arranges the memory buffers for the encoded MPEG stream and sends each stream packet to the Network.
• Decoder: It arranges the memory buffers for the received MPEG stream and feeds it to the decoder card. It also sets the MPEG decoding parameters and controls the decoder card to decompresses the multiplexed MPEG stream to video stream and audio stream and send them to Video displayer module and output ports. In addition, it plays, pulses, step stills and stops the MPEG file stored in disk.

This component serves as a network administrator, which controls the incoming client connection and creates related threads to communicate with the outside world. There are two modules in this component. They are Outgoing module, and Incoming module.

• Outgoing module: there are three blocks in this module, the Outgoing server, the UDP thread and the TCP thread. The Outgoing server block (Figure 4-2) supervises the network condition and responds to the incoming connection requests by either creating two threads (UDP and TCP) to communicate with it or it refuses the connection. It also will cancel the communicating threads if either end want to terminate the communication or a fatal error occurs. The UDP out thread block is a terminal to send the MPEG packets to the network whereas the TCP out thread block is a terminal to send the control signal or chat message to the network.

• Incoming module: it receives the incoming signals from the network and distributes them to the proper modules. The Incoming server block creates both UDP and TCP client end point threads and cancels them if either end want to terminate the communication or a fatal error occurs. The UDP in thread receives the incoming MPEG stream and feeds it to the Decode controller module. The TCP in thread block receives the incoming control signals and chat messages and passes them to the Data distributor block. Whereas the Data distributor separates the two different data and passes them to the Video displayer module and Chat manager module respectively.