The Viper Solution: A Data Persistence Model using XML and PHP

XMLContent

As mentioned before, a web developer encounters XML data in many forms, such as web services messages, RSS/Atom feeds, and/or configuration files. Once he/she decides that this XML data needs to be saved somewhere, we will assume several things about this data. First, we need a way to uniquely identify it, so that once it is saved, it can easily be programmatically found and retrieved (though the API will contain some basic search capability which will be discussed below). The unique name can be a numeric id or a user-provided name.

The second assumption is that the XML data will be stored as is. It will not be modified or changed in any way. Any modifications that are required will be done by the application outside of the XCS. Internally, the XML data is stored as a DOM document, but an application is free to access the data as a file stream, string, or several other convenient access methods which may or may not be implementation-dependent.

Third, the capability will be provided to add additional metadata about the XML data that is saved. For example, if the XML data is a blog entry, perhaps the application would care to know the date and title of the entry, or the hostname where the entry originated. This metadata is saved in an about property and is also XML.

Finally, often times XML data will be accompanied by binary data, such as .jpeg, .pdf, or .doc files. An attachment property will associate this binary data with the XML data. In this first implementation of XMLContent, attachment can contain either 0 or at most 1 item, though optionally, a future version can contain any number of items.

These properties of the XML data are encapsulated in an object called XMLContent. XMLContent objects are the fundamental components of the XCS as they are the XCS representation of XML data. As we will see below, the XMLContentStore component needs to know about the persistence technology (for example, a relational database) and how to access it, but XMLContent objects need not know anything about it.

XMLContentStore

The XMLContentStore represents a repository of XML documents. It is responsible for updating the data source based on changes made to an XMLContent object in the repository as well as retrieving XMLContent objects from the data source based on search or id criteria.

A data source is defined very generally as the persistence layer where the XML data is saved. It can be a relational database, an XML database, or a file system and it stores the XML data in its own format. When an XMLContentStore object is instantiated, it receives a “connection handle” which describes in a meaningful way what the data source is.

An XMLContentStore knows how to access the persistence layer to perform CRUD operations and abstracts these operations by using convenience methods. For example, if the persistence layer is a relational database, a call to the method insert() will build an appropriate SQL insert statement based on the structure of the underlying tables used and the content of the XMLContent object. It will connect to the database and execute the statement. Other CRUD methods work in a similar fashion and include: update(), delete(), deleteById(), and selectAll().

In addition, XMLContentStore also contains a simple search facility that retrieves XMLContent by its id or by searching within the XML data or the about data in XMLContent. The search on XML data is done using XPath expressions. These methods are find(), and findById().

The following classes are helper classes that allow for easier processing of XMLContent objects.

XMLIterator

It is possible that a search returns one XMLContent or a set of XMLContent objects. In the case where a set is returned, the XMLIterator class is used to iterate over the set of XML documents that meet the search criteria. An XMLIterator knows several essential things about the set of XMLContent objects over which it is iterating. These include its current location in the set, how to retrieve the next object in the set, how to go back to the beginning of the set, and when it has reached the last item in the set. This allows the developer to assign behavior on the XML data at each iteration without having to worry about the details of loop control.

XMLUtil

XMLUtil is a utility class that provides convenience methods for passing XML data back and forth from the application to XMLContent, either for the raw XML data or for the about XML metadata. Though the XML is stored internally as a DOM, XMLUtil can allow an application to use strings, file streams, or any other implementation-specific object representation of XML data. Convenience methods for converting between these different types of representations and DOM are provided.

PHP

PHP is a dynamic language that is easy to use and learn. It started out as a small open source project that evolved as more and more people discovered how useful it was. PHP is a scripting language that is embedded in HTML and interpreted by the server. It can be used to manage dynamic content, work with databases, handle session tracking, and even build entire e-commerce sites. It works well with a number of popular databases, include MySQL, Oracle, SQL Server, and DB2 [14].

PHP is an integral part of the LAMP architecture that has become popular in the Web industry as a way of deploying inexpensive, reliable, scalable, and secure web applications. LAMP is short for Linux, Apache, MySQL, PHP, all of which are open-source products [15]. The PHP model can be seen as an alternative to Microsoft’s ASP .NET [16], ColdFusion [17], or Java/JSP [18].

One of the many useful data structures available in PHP that was especially useful in the development of the XCS is the array. In PHP, an array is actually an ordered map, also called an associative array, which maps values to keys. This allows the array data type to be very versatile and to be easily used as an array, list or vector, hash table, dictionary, collection, stack, queue, or tree [19].

Finally, basic object-oriented features to PHP were added in version 3. The latest release, version 5 released in July 2004, contains enhanced and more robust and powerful features to allow PHP to be used as a full-fledged object-oriented language.

Because it is so easy to learn and use (and because it is free), PHP has become the dominate language for serving dynamic web content. As of April 2006, there were over 20,000,000 domains and 1,200,000 IP addresses using PHP [20].

DB2 Viper

DB2 Viper, developed by IBM and now in beta, is the next version of the DB2 database management system. It features significant new support for storing, managing, and searching XML data, in addition to its traditional relational database management scheme. Viper allows client applications to work with both tabular and XML data structures through the query language of their choice – SQL (including SQL with XML extensions, often referred to as "SQL/XML") or XQuery. As shown in Figure 1, engine-level components within DB2 support queries specified in either language [21].

Figure 1. The architecture of the new DB2 "Viper" release

XML is treated as a first class data type in Viper and a table can be defined to contain a column of type XML. This familiar syntax and usage allows quick adoption and integration with existing relational models and applications. Still, a purely XML-centric developer can simply stick to the XQuery language to manipulate XML data structures within Viper.

To efficiently manage XML data like traditional SQL data types, the XML data model is the fundamental logical data model both used internally by the database and exposed to database users when XML is the data type. Also, the XML data model is the fundamental unit of physical storage. However, it's important to note that the underlying storage mechanism used for a given data type is transparent to the application. In other words, the application doesn't need to explicitly specify which storage mechanism to use or to manage physical aspects of storage, such as splitting portions of XML documents across multiple database pages. It simply enjoys the runtime performance benefits of storing and querying data in a format that's efficient for the target data[22] [23].

Because of its native XML capabilities, Viper is a natural choice for the persistence layer for the XCS.

XCS using PHP and Viper

For the Viper implementation of the XCS, we defined one table with four columns that represent the properties of an XMLContent object that was described above: id, data, about, and attachment.

The columns are defined as follows:

• id is a unique integer and used as the primary key of the table

• data is defined as an XML column

• about is defined as an XML column

• attachment is defined as a BLOB column

Using Viper, each row in the table represents one XMLContent. Creating the table is the only database administrative task required. Yet, this can also be automated by adding the appropriate implementation-specific code in the API. This one-time enabling of Viper for the XCS by creating the table can then be executed via a programmatic method call. Figure 2 is a visual representation of the database table used for the XCS.

Figure 2. Using DB2 Viper as the persistence layer for the XCS

As we mentioned above, XMLContent is not aware of what the persistence mechanism is, so it can be used with Viper or any other implementation of the XCS without any changes. PHP contains a set of classes that allows the developer to operate on an XML document with the standardized DOM API and is fully object-oriented. Internally, the data and about properties are stored as PHP DOMDocument objects.

PHP also contains a simple XML representation called SimpleXML. It provides a very simple and usable object the can be processed with normal PHP property selectors and array iterators [24]. It is much easier when using simple, small XML documents to work with the SimpleXML class than with the DOMDocument class. For this reason, convenience methods for SimpleXML were added to XMLUTil so that an application may choose to do all its work in SimpleXML, but would also be able to use the XCS.

Since XMLContentStore is required to know about the persistence layer, it will be unique for each implementation. For its use with Viper, we created a child class of XMLContentStore called XMLContentStoreDB2. We implemented the CRUD methods using the PHP CLI driver, named ibm_db2. ibm_db2 provides a set of functions that enable applications to access the IBM DB2 Universal Database, IBM Cloudscape, and Apache Derby [25]. ibm_db2 uses connection, statement, and resultset resources and passes SQL directly to the database. XMLContentStoreDB2 abstracts all these details to make developer interaction with the persistence layer simple and fast.

It is important to note that the Viper implementation was simple because Viper allows us to treat XML as a first-class data type. By allowing us to create columns of XML type, the CRUD operations were easily implemented using simple SQL statements. There was no need to map XML elements to SQL columns or to perform any other type of object-relational mapping.

DB2 Viper Database Definition

To create the XCS database in DB2, the following Data Definition Language (DDL) can be used. This DDL is executed at the DB2 command line or from a script where the commands are placed.

// Viper uses an utf-8 database to support XML
db2 create database xmlrepos USING CODESET utf-8 TERRITORY us;
// create the table
db2 create table xmldata(
   id bigint not null generated always as identity primary key,
   data xml,
   attachment blob(100m),
   about xml
);

The XML data type is available on any utf-8 database in the DB2 Viper release. With the creation of these database objects, DB2 is now ready to be used as an XML Content Store. There are no other DB2 administrative tasks necessary. However, to improve performance when searching, we suggest that indexes be created for the XML data:

create index datatext on xmldata(data) generate key using xmlpattern '//*' as sql varchar(800)
create index dataattr on xmldata(data) generate key using xmlpattern '//@*' as sql varchar(800)

create index abouttext on xmldata(about) generate key using xmlpattern '//*' as sql varchar(800)
create index aboutattr on xmldata(about) generate key using xmlpattern '//@*' as sql varchar(800) 

These are very general indexes on all text nodes and attributes on both data and about. The developer may decide to create indexes on elements or attributes for application-specific data, such as a product id or a last name element where searches may occur often.

XCS Implementation Example

The class XMLContentStoreDB2 is implemented using the ibm_db2 CLI driver. For example, to insert, the application gives the XCS an XMLContent object and the XMLContentStoreDB2 takes care of the underlying details:

/*** Saves the XML data represented by an
* XMLContent object into the persistence layer
*
* @param XMLContent $doc
*
* @return integer
*/
public function insert(XMLContent $doc)
{
   // SELECT FROM INSERT is a DB2 construct that allows
   // an application to immediately select values after
   // an insert. In this case we are retrieving the id
   // generated for the document using the sequence
   // database object
   $insertStr = 'SELECT id FROM FINAL TABLE(' .
                'INSERT INTO ' . 
                'xmldata(data,attachment,about)' .
                'VALUES(?,?,?))';

   // prepare the statement   
   $stmt = db2_prepare($this->conn, $insertStr);
   $xmlDoc = $doc->getDOM();   
    
   // set parameters   
   $paramArray = array($xmlDoc,
                       $doc->getAttachment(),
                       $doc->getAbout()->saveXML());

   // execute    
   db2_execute($stmt, $paramArray);

   // fetch results    
   $idArr = db2_fetch_array($stmt);
   $id = $idArr[0];
   $doc->setId($id);

   // return the id
   return $id;
}

The other CRUD and find methods are implemented in a similar fashion.

Example of an Application’s Use of the XCS

An application can easily manipulate and persist XML data using the XCS. First, instantiate the XMLContentStoreDB2 object, assuming that a connection resource has been previously created:

// Construct the XMLContentStoreDB2 object
// We assume that $conn is a connection resource to a DB2
// database that has already been created.
$xmlstore = new XMLContentStoreDB2($conn);

Perform an insert into the XCS:

// createDocument() is a convenience method in XMLUtil
// to create an XMLContent object
$doc = XMLUtil::createDocument();
// use a utility method to import a file into
// the XMLContent
XMLUtil::importXMLFile(‘data.xml’);
$xmlstore->insert($doc);

Let’s do a find as well. Assume a set of blog entries are stored in the XCS. A sample entry might look like this:

<?xml version="1.0" encoding="windows-1252" ?>
<entry>
   <title>Elvis Lives</title>
   <author>John Doe</author>
   <date>11-22-2003</date>
   <body>I recently wrote a short history of the works of Elvis Presley</body>
</entry>

To perform the search, one might issue the following statements:

$lookingFor = ‘elvis’;
$found = $xmlstore->find($lookingFor);
// iterate over possibly many results
foreach($found as $current) {
   // do something with the result
}

In this case, the above blog entry would be returned because the string “elvis” was a match in the <title> element and the <body> element (though only one match is needed in order to determine search success). In this example, the search was case-insensitive, but the find() method can also take a parameter indicating that the case must also match.

Typical XCS Applications

There are many types of applications that will benefit by using the XCS. Figure 3 demonstrates the basic architecture of an application that uses the XCS.

Figure 3: The architecture of a web application using the XCS

Below we list some specific application examples.