BOT-Technology and Real-Time Databases

Consistent Utilization of Indexes

In the area of "Process Control" data processing, in which timely, complex transactions need to be processed, the usual "Big" Databases have only limited application.

By means of a Laboratory Report on the CodeBase database engine, we explore the features of Open, real-time database systems in the areas of Data Storage and Query Optimization.

Most of the Information Systems in operational use today are based on classic query processing, where the DBMS does not generally have tight time-constraints. With these kinds of systems the primary consideration is database consistency over transaction throughput rate.

The situation is quite different with "Process Control" data processing however, where the time-critical control and regulation of plant and equipment is the prime consideration. If we want to control furnaces, plant and satellites then we need to process time-constrained transactions. This is the only way that we can guarantee that the database maps correctly to the environmental data, that may for example be input using sensor equipment.

Modern relational database systems offer the advantages of extensions e.g. "Triggers" where these are offered as key features, but these extensions generally place a burden on performance. Real-time system developers thus have no choice, other than to use a fast, high performance real-time database system. With the CodeBase product we are presenting here, we will show how "open", real-time database systems can be developed.

CodeBase is produced by Sequiter Software, and has been available since 1988. The original version consisted of only a ‘C’ library for developing standalone DOS database applications. In the latest version there is both standalone and client/server configurations, with language support for C, C++, Visual, Delphi and Java. The Xbase standard is used as the internal schema for data storage. The platforms supported range from DOS, Windows, OS/2, and Macintosh to Intel-Based UNIX systems (e.g. SCO, Linux) and Non-Intel systems such as AIX, Solaris and so on. A noteworthy feature is that the full source code comes with the system as part of the standard package. Along with the pre-compiled class-libraries and DLLs, together with source code, Sequiter includes the CodeControls developer tools and CodeReporter. CodeControls is a set of VBX and OCX data-aware controls that ease the creation of data-centric forms under Windows, while CodeReporter lets you create reports interactively.

Sequiter’s advertising, which positions CodeBase in the market as a high performance database, will catch peoples' attention. But just how fast is the system? In order to test the performance of CodeBase, as a first step a benchmark was developed, which generates a database with a simple relation. The insertion of a row (Append) into the database was selected as a "critical" operation, usually one of the most time-critical transactions in the operational world. This Benchmark is to establish how much data (e.g. values entered from sensor equipment) per time-unit can be entered into the database within a given time unit.

Although a benchmark can ascertain a system's performance using specific workloads, this doesn’t necessarily give us any performance information regarding concurrent systems. For this purpose the Jet Engine was used as reference system for the Append Benchmark. Thus a comparative statement of performance is possible. The performance measurement of the Append Benchmark was taken for 1000, 5000, 10000,..., 50000 records. The individual results and the derived term Transaction/Second are shown in Table 1.

The higher performance that CodeBase shows over the Jet-Engine in append operations can be traced back to the internal buffer management used by the engine, which doesn’t implement page-writes on a record basis, but rather on a block basis. Analysis of the term Transactions/Second shows an increasing burden in the performance of the Jet-Engine, while the performance of CodeBase is stable.

For fast applications, after data storage considerations, performance of queries is almost as important. For this, CodeBase supports query optimization via Bit Optimization Technology (BOT). This concept is used automatically by the internal CodeBase query manager to optimize queries and is completely transparent to the programmer. With every query, the BOT-Technology uses all of the corresponding available index information in order to determine the query result.

When executing a query, all of the data rows in the database are represented in a bit-field, in which all records selected by the query are represented by a 1, and all other records are represented by a zero. The installation of Bit Optimization Technology depends on the application type. For 16-bit applications, a maximum of. 500,000 records can be queried using this technique, as the bit field has a maximum size of 64K. For 32-Bit applications with an address range of 4 GB, the technology has a potential maximum of 32 billion records.

With CodeBase, developers can create scalable solutions that are single user, multi-user (via file locking), or client/server. The Server Module runs based on TCP/IP under Windows 95 and Windows NT. The total number of client connections is not limited so that in principle there is no limit to the number of users that can connect to the CodeBase server.

A real problem for many data processing departments is the development of transaction-based Web applications. It is not just about presenting static HTML pages any more, but rather more about integrating the customers into the business process of the company. This will only be achieved when a genuine Online Transaction Processing (Internet Transaction Processing, ITP) is used, and incoming orders do not first have to be printed out, in order to process those orders further. Orders, coming in from the Internet, could be connected directly to the Customer and Sales Order administration and availability systems.

CodeBase has been ported to Java for the development of integrated Web databases. In the delivery range is a Java Class Library that sits in the source code and gets translated/interpreted by the Java Compiler from the JDK. It should be stressed that a critical factor in the download for the end-user is the size of the generated Java classes. At close to 50 KB this can best be described as "compact". Since the internal dependencies of the class library are documented, the developer is able to remove any unusable functions from the CodeBase Java classes in order to reduce the library size and thus decrease download times. Therefore with pure reading access onto a Web database, the stripping of the Locking and Create functions is recommended. Figure 3 shows an example of how a Java Applet can create an indexed database with the two fields "Name" and "Age".

The advantage that the developer has with CodeBase is as follows: By using the Xbase standard as internal storage schema, it is guaranteed that data from all other modules in the application can be further processed.

	import codebase.*;
	import java.io.IOException;
	import java.net.UnknownHostException;

	class CodeBaseExample {
	  public static void main(String args[])
	    throws Error4,IOException,UnknownHostException {

	    Code4 client = new Code4();
	    // Create connection

	    client.connect(null, -1, "user1", "");
	 
	    // Create new database object
	    Data4 newDataFile = new Data4(client);

	    // Fields NAME and AGE for creating table
	    Field4info fieldInfo = new Field4info();
	    fieldInfo.add("Name", Field4info.CHARACTER, 10, 0);
	    fieldInfo.add("Age", Field4info.NUMERIC, 3, 0);

	    // Create data file
	    newDataFile.create("PERSONS", fieldInfo);
	    System.out.println("Persons data file was created");

	    // Create index for NAME
	    Tag4info tags = new Tag4info();
	    tags.add("NameTag","Name",null,Tag4info.DUPLICATE,false);
	    newDataFile.createIndex( null, tags);
	    System.out.println("Name index was created");

	    // Create index for AGE
	    Tag4info newTags = new Tag4info();
	    newTags.add("AgeTag","Older",null,Tag4info.DUPLICATE,false);
	    newDataFile.createIndex("Ageindex", newTags);
	    System.out.println("Age index was created");

	    newDataFile.close();
	  }
	}
	

For the development of an appropriate "Look & Feel" for database forms, CodeBase provides its "CodeControls". These data-bound ActiveX Controls consist of a series of dialog elements that make input and output as well as database navigation easier. Fig. 4 gives some examples of the controls available.

Fig. 4: CodeControls in operation

Development environments such as Visual Basic have the disadvantage that formatted input and output of data needs to be organized in specific ways. Using CodeControls can considerably reduce the problem, because the formatting options for input and output are very extensive indeed. The CodeReporter program is a report generator that allows the design and creation of relational reports. A Preview function enhances report writing. An interactively developed report can be run in two ways.

First, a report can be stored as encoded report data, and then at run-time opened and interpreted using special CodeBase Functions. This strategy would be recommended where a user creates or modifies reports, without having to recompile the entire application. Alternatively the corresponding source code for the report can be generated. This would be connected to the Project and activated by means of function calls.

A Practical Case Study...

CodeBase was used to develop a database for the Westfalian Wilhelms University Of Munster to control licenses in the business data processing arena. The license control system identifies and controls login and program activities on all of the 450 processors on a Novell network in real-time. In the event of license breaches, the corresponding program is locked until another user releases. Since this leads to many tightly bundled requests with this application, the demands on response time and database performance are unusually high. The license control system, known as "User Access Monitor", running on a separate processor under DOS, processed 1.18 million network transactions in 1996. The data that was collected in this way are analyzed using OLAP devices and form the basis of strategic license planning.

Fig. 5: License Control Database design

This short extract shows that the CodeBase product is an attractive component for the development of fast databases for classical LANs as well as for the Internet. The price/performance ratio, the open availability of source code, and its Internet capabilities make CodeBase an economical and high performing database solution.

Author Information:

Dipl.-Kfm. Frank Bensberg
Westfalian (Wilhelms) University
Institut für Wirtschaftsinformatik
Grevener Str. 91
48159 Münster

Frank Bensberg works as an assistant lecturer for Business Computer Science and Control,Westfalian (Wilhelms) University Of Munster. His speciality areas are multimedia systems, network license control, OLAP and EIS.