Transform plugins examine or transform HTTP message body content. For example, transform plugins can:

• Append text to HTML documents

• Compress images

• Do virus checking (on client POST data or server response data)

• Do content-based filtering (filter out HTML documents that contain certain terms or expressions)

This chapter explains how to write transform plugins. The following examples are discussed in detail:

• Sample Null Transform Plugin
• Append-Transform Plugin
• Sample Buffered Null Transform Plugin

Writing Content Transform Plugins

Content transformation plugins transform HTTP response content (such as images or HTML documents) and HTTP request content (such as client POST data). Because the data stream to be transformed is of variable length, these plugins must use a mechanism that passes data from buffer to buffer and checks to see if the end of the data stream is reached. This mechanism is provided by virtual connections (VConnections) and virtual IO descriptors (VIOs).

A VConnection is an abstraction for a data pipe that allows its users to perform asynchronous reads and writes without knowing the underlying implementation. A transformation is a specific type of VConnection. A transformation connects an input data source and an output data sink; this feature enables it to view and modify all the data passing through it.

Transformations can be chained together, one after the other, so that multiple transformations can be performed on the same content. The VConnection type, TSVConn, is actually a subclass of TSCont, which means that VConnections (and transformations) are continuations. VConnections and transformations can thus exchange events, informing one another that data is available for reading or writing, or that the end of a data stream is reached.

A VIO is a description of an IO operation that is in progress. Every VConnection has an associated input VIO and an associated output VIO. When VConnections are transferring data to one another, one VConnection's input VIO is another VConnection's output VIO. A VConnection's input VIO is also called its write VIO because the input VIO refers to a write operation performed on the VConnection itself. Similarly, the outpt VIO is also called the read VIO. For transformations, which are designed to pass data in one direction, you can picture the relationship between the transformation VConnection and its VIOs as follows:

Transformation and its VIOs

Because the Traffic Server API places transformations directly in the response or request data stream, the transformation VConnection is responsible only for reading the data from the input buffer, transforming it, and then writing it to the output buffer. The upstream VConnection writes the incoming data to the transformation's input buffer. In the figure above, A Transformation and its VIOs, the input VIO describes the progress of the upstream VConnection's write operation on the transformation, while the output VIO describes the progress of the transformation's write operation on the output (downstream) VConnection. The nbytes value in the VIO is the total number of bytes to be written. The ndone value is the current progress, or the number of bytes that have been written at a specific point in time.

When writing a transformation plugin, you must understand implementation as well as the use of VConnections. The implementor's side refers to how to implement a VConnection that others can use. At minimum, a transform plugin creates a transformation that sits in the data stream and must be able to handle the events that the upstream and downstream VConnections send to it. The user's side refers to how to use a VConnection to read or write data. At the very least, transformations output (write) data.

Transformations

VIOs

A VIO or virtual IO is a description of an in progress IO operation. The VIO data structure is used by VConnection users to determine how much progress has been made on a particular IO operation, and to reenable an IO operation when it stalls due to buffer space. VConnection implementors use VIOs to determine the buffer for an IO operation, how much work to do on the IO operation, and which continuation to call back when progress on the IO operation is made.

The TSVIO data structure itself is opaque, but it might have been defined as follows:

typedef struct {
     TSCont continuation;
     TSVConn vconnection;
     TSIOBufferReader reader;
     TSMutex mutex;
     int nbytes;
     int ndone;
} *TSVIO;

IO Buffers

The IO buffer data structure is the building block of the VConnection abstraction. An IO buffer is composed of a list of buffer blocks which, in turn, point to buffer data. Both the buffer block (TSIOBufferBlock) and buffer data (TSIOBufferData) data structures are reference counted so they can reside in multiple buffers at the same time. This makes it extremely efficient to copy data from one IO buffer to another using TSIOBufferCopy, since Traffic Server only needs to copy pointers and adjust reference counts appropriately (instead of actually copying any data).

The IO buffer abstraction provides for a single writer and multiple readers. In order for the readers to have no knowledge of each other, they manipulate IO buffers through theTSIOBufferReader data structure. Since only a single writer is allowed, there is no corresponding TSIOBufferWriter data structure. The writer simply modifies the IO buffer directly.