Implementation of a declarative query and data manipulation language for X-Ray

Hiebl, Christian

Implementation of a declarative query and data manipulation language for X-Ray

Zusammenfassung

Inhaltsangabe:Abstract:
Nowadays, database management systems (DBMS) play a central role in the realization of modern information systems for efficient storage, management and retrieval of large amount of data. At the same time, the eXtensible Markup Language (XML) is emerging fast as de facto standard for electronic data exchange. In order to be able to benefit from both technologies, a number of approaches have already been developed, aiming at the integration of XML and DBMS. They allow processing XML data on the basis of declarative query languages, dont pay however much attention to the manipulation of XML data.
The objective of this diploma thesis is to implement X-Ray QL, a declarative query and data manipulation language for X-Ray, an integration approach using the idea of a meta database. X-Ray QL allows besides creating, editing and deleting XML data in a declarative way, to retrieve the XML data using simple select queries.
The implementation of X-Ray QL consequently provides an operative runtime part of the X-Ray architecture with features like session management, authorisation mechanism and transaction control. An additional Web Service demonstrates the practical use of the X-Ray QL implementation.

Zusammenfassung:
Datenbankmanagementsysteme (DBMS) spielen heutzutage eine zentrale Rolle bei der Realisierung moderner Informationssysteme zur effizienten Speicherung, Verwaltung und Verarbeitung großer Datenmengen. Gleichzeitig steht mit der eXtensible Markup Language (XML) eine erweiterbare Auszeichnungssprache zur Verfügung, die unter anderem als Quasi-Standard für den elektronischen Datenaustausch gilt. Um die Vorteile beider Technologien nutzen zu können, existiert bereits eine Reihe von Ansätzen, die eine Integration von XML und DBMS zum Ziel haben. Diese Ansätze erlauben es XML Datenbestände auf Basis deklarativer Abfragesprachen zu verarbeiten, jedoch wurde das Verändern von bestehenden Datenbeständen noch nicht genauer behandelt ([KIM02]).
Ziel dieser Diplomarbeit ist es X-Ray QL, eine deklarative Abfrage- und Datenmanipulationssprache für X-Ray, ein Integrationsansatz unter Verwendung einer Metadatenbank, zu implementieren. X-Ray QL ermöglicht, neben dem deklarativen Erzeugen, Ändern und Löschen von XML Datenbeständen, auch einfache Abfragen durchzuführen.
Durch die Implementierung von X-Ray QL entsteht der Laufzeitteil der X-Ray Architektur, welcher sich durch Sessionmanagement, Autorisierungsmechanismus und […]

Leseprobe

Inhaltsverzeichnis

ID 7580

Hiebl, Christian: Implementation of a declarative query and data manipulation language

for X-Ray

Hamburg: Diplomica GmbH, 2004

Zugl.: Johannes Kepler Universität Linz, Universität, Diplomarbeit, 2002

Dieses Werk ist urheberrechtlich geschützt. Die dadurch begründeten Rechte,

insbesondere die der Übersetzung, des Nachdrucks, des Vortrags, der Entnahme von

Abbildungen und Tabellen, der Funksendung, der Mikroverfilmung oder der

Vervielfältigung auf anderen Wegen und der Speicherung in Datenverarbeitungsanlagen,

bleiben, auch bei nur auszugsweiser Verwertung, vorbehalten. Eine Vervielfältigung

dieses Werkes oder von Teilen dieses Werkes ist auch im Einzelfall nur in den Grenzen

der gesetzlichen Bestimmungen des Urheberrechtsgesetzes der Bundesrepublik

Deutschland in der jeweils geltenden Fassung zulässig. Sie ist grundsätzlich

vergütungspflichtig. Zuwiderhandlungen unterliegen den Strafbestimmungen des

Urheberrechtes.

Die Wiedergabe von Gebrauchsnamen, Handelsnamen, Warenbezeichnungen usw. in

diesem Werk berechtigt auch ohne besondere Kennzeichnung nicht zu der Annahme,

dass solche Namen im Sinne der Warenzeichen- und Markenschutz-Gesetzgebung als frei

zu betrachten wären und daher von jedermann benutzt werden dürften.

Die Informationen in diesem Werk wurden mit Sorgfalt erarbeitet. Dennoch können

Fehler nicht vollständig ausgeschlossen werden, und die Diplomarbeiten Agentur, die

Autoren oder Übersetzer übernehmen keine juristische Verantwortung oder irgendeine

Haftung für evtl. verbliebene fehlerhafte Angaben und deren Folgen.

Diplomica GmbH

http://www.diplom.de, Hamburg 2004

Printed in Germany

Table of Contents

Introduction ... 9

1.1 Motivation ... 9

1.2

Goals of this Thesis ... 10

1.3

Structure of this Thesis... 11

X-Ray Architecture ... 12

2.1 Motivation ... 12

2.2

X-Ray Architecture in Detail ... 12

2.2.1

The Definition Part... 13

2.2.2

The Runtime Part ... 14

2.2.3

The Meta Database Interface... 15

2.3 Architectural

Terminology... 16

2.3.1 X-Ray

Source ... 16

2.3.2 XML

Document ... 16

2.3.3 Repository ... 18

X-Ray Runtime Implementation ... 23

3.1 Authorisation... 23

3.1.1 Motivation ... 23

3.1.2 Authorisation

Engine... 23

3.2 Session

Management... 25

3.3 QL

Engine ... 27

3.3.1

X-Ray QL Query Execution... 28

3.3.2

X-Ray QL Query Verification... 31

3.4 Syntax

Check... 31

3.5 XPath

Engine... 34

3.6 SQL

Engine ... 37

3.7 SQL

Queries... 38

3.7.1 Select

Query ... 39

3.7.2 Update

Query ... 41

3.7.3 Insert

Query... 42

3.7.4 Delete

Query ... 42

3.8 Select

Engine... 42

3.8.1 Initialisation... 43

3.8.2 Execution... 44

3.8.2.1 Target

Attributes ... 45

3.8.2.2 Query

Source... 48

3.8.2.3 Predicates ... 49

3.8.2.3.1 Predicates with Position Information ... 50

3.8.2.3.2 Predicates without Position Information ... 51

3.8.2.4 Query

Processing... 51

3.8.2.5 Query

Result... 52

3.9 Update

Engine ... 52

3.9.1 Simple

Update ... 53

3.9.2 Cascading

Update... 54

3.10 Insert

Engine ... 56

3.10.1 Direct

Insert

Operations ... 56

3.10.2 Indirect

Insert

Operation ... 58

3.11 Delete

Engine ... 60

3.11.1

Direct Delete Operation ... 60

3.11.2

Indirect Delete Operation ... 61

3.12 Transaction

Control... 62

3.12.1 Commit... 62

3.12.2 Rollback ... 64

X-Ray Language Definition ... 65

4.1

Commit / Rollback ... 66

4.2 Select ... 66

4.2.1 XPath

Expression ... 66

4.2.2 X-Ray

Source ... 68

4.3 Update ... 69

4.4 Insert... 69

4.5 Delete ... 73

X-Ray QL Web Service ... 75

5.1

X-Ray QL Deployment Descriptor ... 75

5.2

Web Service Client... 77

Outlook ... 81

6.1

Extension of the X-Ray Runtime Part... 81

6.1.1 Authorisation

Engine... 81

6.1.2 Document-oriented

Data ... 82

6.2

Extension of the Web Service Client ... 82

6.3

Extension of X-Ray QL... 83

6.3.1

X-Ray Architecture Extensions... 83

6.3.2

XML Mixed Content Types ... 84

6.3.3

X-Ray QL Rename Operation... 84

6.3.4

X-Ray QL Assign Operation... 84

Conclusion... 86

Index of Figures... 92

Index of Tables... 94

Index of Examples... 96

Appendix A: X-Ray QL ... 99

Appendix B: CoCo Input Languages ... 102

Appendix C: Example ... 110

Appendix D: X-Ray Definition Part Extensions... 135

Lebenslauf ... 136

Chapter 1

Introduction

1.1 Motivation

The eXtensible Markup Language (XML) [BRA00] is a universal format for structured

documents and data on the Web, which has been established in many different application

areas. XML is the basis for a lot of other standards, for example the ebXML (Electronic

Business using eXtensible Markup Language) [EBX01] standard, which is used to define

business processes and their associated messages and content, for a global electronic market.

Nowadays XML and related technologies like XSL (eXtensible Stylesheet Language), are

used to manage web content and content presentation on the World Wide Web. XSL is a

language for expressing style sheets, which consists of three parts: XSL Transformations

(XSLT) [XSLT99], a language for transforming XML documents, the XML Path Language

(XPath) [XPA99], an expression language used by XSLT to access or refer to parts of an

XML document, and XSL Formatting Objects, an XML vocabulary for specifying formatting

semantics.

Although XSL may be used for content formatting, the web application's data is still stored

within database systems. Meanwhile, the database research community has been hard at work

in addressing the challenges of providing XML views of relational databases, with systems

such as SilkRoute [SIL00] and XPERANTO [XPE00]. Another development on the database

vendor sector are native XML database systems (e.g. Tamino Server [TAM01]), which use a

special internal schema for storing XML data. Other database vendors provided their database

systems with functionalities for writing and administering XML data

[ORA01][IBM01][MIC01], which lead to the integration of XML and relational databases.

In this way increasing amounts of information are stored, exchanged, and presented using

XML, and the ability to intelligently query XML data sources becomes increasingly

Introduction

Page

important. One of the great strengths of XML is its flexibility in representing many different

kinds of information from diverse sources. To exploit this flexibility, XML query languages

[BON99][FER01] were introduced, which provide features for retrieving and interpreting

information from these diverse sources. Examples for XML query languages are Quilt

[QUI00], XQuery [XQU02], XQL [XQL99], XMLQL [DEU99] and Lorel [LOR97].

The next step in making XML into a full-featured data exchange format is to support not only

queries, but also updates, over XML content stored within relational databases. It should be

possible to modify content within XML documents and to express updates to XML views,

which represent the XML data contained in database systems, and which are percolated back

to the original data [TAT01].

There are only a few approaches for designing an XML update language. One of them,

XUpdate [XUP00], provides open and flexible update facilities to modify data in XML

documents. This approach makes location of documents transparent. It can be real (i.e.

materialised) documents or virtual documents retrieved from XML databases. An update in

the XUpdate language is expressed as a well-formed XML document. XUpdate makes

extensive use of the expression language defined by XPath for selecting elements for updating

and for conditional processing, and is a pure descriptive language, which is designed with

references to the definition of XSL Transformations (XSLT).

Lexus is the Java based reference implementation of the XUpdate specification that is used in

several Open Source projects and commercial products. The current Lexus version is

available for download at http://www.xmldb.org, the organisation developing Lexus and the

XUpdate standard.

1.2

Goals of this Thesis

Goal of this diploma thesis is to implement X-Ray QL, which was developed as update

language and as simple query language for XML. This query language was explicitly

designed for the X-Ray architecture [ART99][OBE01], which supports the integration of

data-oriented XML data models and relational database systems.

The basic concepts behind X-Ray QL have been already described in [KIM02].

The following points have to be considered during the implementation of X-Ray QL:

- Extension and completion of the existing X-Ray QL definition [KIM02]

- Implementation of X-Ray QL with respect to the extension of the meta database

[OBE02] in form of a class library

- Integration of the X-Ray definition part (diploma thesis of Martin Oberhammer

[OBE02]) and the runtime part in a common architecture (as described in chapter 2)

- Design and Implementation of an interface to the runtime part of the X-Ray

Introduction

Page

architecture (for example as a Web Service) as described in chapter 5

- Design and Implementation of a graphical user interface for deposing X-Ray QL

queries (as described in section 5.2)

1.3

Structure of this Thesis

At the beginning of this diploma thesis the actual X-Ray architecture will be described in

short terms for a better understanding of the main concepts. This comprises a short

explanation of the definition part's architecture and an overview over the main components of

the runtime part. The meta database interface is another subject of this chapter. The basic

terms of the X-Ray runtime part will be introduced in a section over the architectural

terminology.

The detailed documentation of the X-Ray runtime part can be found in chapter 3, where all

main components of the current architecture implementation are described in detail. For a

better understanding this chapter is exemplified with adequate code and usage examples. This

chapter documents the core implementation work of this diploma thesis.

Chapter 4 is dedicated to the X-Ray QL syntax as implemented by the X-Ray runtime part.

This chapter can be seen as a user guide for the X-Ray query language and is therefore

completed with a number of examples.

The user interface to the X-Ray runtime part is described in chapter 5, which proposes a

possible implementation of a Web Service and a corresponding Web Service client. The Web

Service was implemented as an Apache SOAP server [APA01][JAK01] and the Web Service

client is implemented as a prototypical Java application.

Chapter 6 will give an outlook at further research and development of the topic X-Ray query

language and X-Ray runtime part. The last chapter is about the conclusions drawn from the

work on this diploma thesis, containing ideas on improving the X-Ray query language, the X-

Ray runtime part and the X-Ray architecture. This chapter also deals with emerged problems

and will complete this diploma thesis.

This diploma thesis is documented with an example on the world's literature history. The

source XML documents, Document Type Definitions and the data contents of the meta

database and relational database are contained in the appendix of this work.

Chapter 2

X-Ray Architecture

This chapter describes the X-Ray Architecture, which has been dealt of in the thesis of

[KAP00] and [ART99]. Therefore this chapter will only introduce the main concepts for a

better understanding of the following chapters of this work.

2.1 Motivation

X-Ray is an approach to integrate XML and relational database systems. X-Ray differentiates

from other approaches by not only offering possibilities for storing XML datasets in relational

database systems, but also by providing a mapping mechanism by the use of a meta database.

This meta database contains mappings between XML schema information (in form of a DTD)

and a relational schema, and consists of database relations stored in a relational database.

The advantage of this approach is, that the DTD and the relational schema do not have to be

changed, and may therefore be used further on with other applications. So the autonomy of

the DTD and the relational database schema will remain unaffected.

With this approach it is possible to interpret data contained in a relational database schema as

a XML document. The user is enabled to query, completely transparent, XML documents

from the relational database schema.

2.2

X-Ray Architecture in Detail

The following figure shows the existing architecture of X-Ray as drafted in the practical of

Eugen Kimmerstorfer and Martin Oberhammer [OBE01].

X-Ray Architecture

Page 13

Figure 1 - X-Ray architecture

The X-Ray architecture consists of two parts:

- Definition

part

- Runtime part (shown as black box)

2.2.1 The Definition Part

The definition part was realised in the diploma thesis of Martin Oberhammer [OBE02] and is

used to insert concepts (these are relational database descriptions and XML Document Type

Definition (DTD)) into the meta database. The X-Ray administrator may administer the

concepts and build mappings between certain concepts. At this the X-Ray init tool developed

by Martin Oberhammer assists the X-Ray administrator. It provides convenience for inserting,

deleting and modifying these concepts and the mapping between them. Generators like the

mapping generator, the schema generator and so on, are responsible to write information

about the concepts into the meta database by accessing the meta database via an interface, the

MetaDBEngine. The meta database was designed at the department of information systems

(IFS) at the Johannes Kepler University of Linz and implemented in Java by Eugen

Kimmerstorfer and Martin Oberhammer during a practical course work. Martin Oberhammer

X-Ray Architecture

Page 14

implemented the X-Ray init tool in the course of his diploma thesis. The definition part is

documented in [OBE02]. For the integration with the X-Ray runtime part the definition part

had to be extended as documented in the appendix.

2.2.2 The Runtime Part

The runtime part was drafted in the diploma thesis of Eugen Kimmerstorfer [KIM02] and its

implementation is the primary focus of this diploma thesis. The main purpose of this part

deals with the creation of XML documents and the modification of existing XML documents.

To simplify the creation and manipulation of XML documents X-Ray QL was introduced, to

enable a user to select, insert, update or delete XML data from or into XML documents

[KIM02]. Chapter 4 enters into the implementation and realisation of X-Ray QL.

With the mapping information from the meta database and the information about XML and

relational concepts the runtime part is able to form XML documents. As the X-Ray query

language is able to manage more than one XML document, the runtime system must be

capable of allocating different values of one database relation to different XML documents.

The realisation of this core functionality is described in section 2.3.2 "XML Document"

below.

Figure 2 - X-Ray runtime part components

The X-Ray runtime part consists of several engines for processing X-Ray QL queries.

The main engine of the X-Ray runtime system is the QL engine, which is responsible for

managing all other engines (the arrows in the figure show the flow of control). It acts as an

entry point for X-Ray QL queries and returns the results of a query processing. As not every

user may access the X-Ray runtime part functionality, the QL engine orders the authorisation

engine to verify the user's identity. The XPath engine takes care of XPath expressions inside

X-Ray QL queries and is therefore responsible for identifying the XML concepts concerned

by an X-Ray QL query. The SQL engines use the identified XML concepts and use the

X-Ray Architecture

Page 15

mapping information from the meta database to retrieve the relational concepts. Then the SQL

engines process SQL queries on the relational databases containing the relational concept

data. These SQL queries may be selects, inserts, updates or deletes that modify in this way the

data of a XML document.

So the runtime system provides access to the XML documents administered by the meta

database. The interface between user and X-Ray runtime part may be implemented in

manifold variants. The user may access the X-Ray runtime system via a Java web application

in a web browser, or a Windows application implemented in Visual Basic, and many more. A

possible implementation is shown in chapter 5.

2.2.3 The Meta Database Interface

Both, the definition part and the runtime part have access to the meta database. The definition

part implements an interface to the meta database, which provides functionality for retrieving

and writing XML concepts and relational database concepts. On the one hand the runtime part

accesses this interface and on the other hand it implements its own meta database interface.

This runtime part meta database interface is realised by the MetaDBAccess class, which

exclusively handles all access to the meta database.

MetaDBAccess

addXMLDocument()

closeConnection()

commit()

delXMLDocument()

getNextDocumentKey()

getRDBConnectInfo()

getRepository()

getUser()

getXMLDocument()

rollback()

isNoYesAttribute()

Figure 3 - MetaDBAccess class

The MetaDBAccess class provides useful functions for retrieving repositories (see later on

in section 2.3.3) and XML documents (section 2.3.2), and for creating and adding XML

documents within the meta database. With the getUser function user information can be

retrieved, to verify a user's identity during authorisation. The commit and rollback functions

support transactional behaviour of the meta database. A certain relational database connection

can be created with the information retrieved by the getRDBConnectInfo function.

X-Ray Architecture

Page 16

2.3 Architectural

Terminology

The following sections describe the different components of the X-Ray runtime part

architecture in detail. However, as every component works on X-Ray QL queries some basics

of X-Ray QL queries, which affect the X-Ray architecture, have to be described first. Other

basic terms like XML concept, relational concept or XML DTD can be looked up in

[KIM02][OBE02].

2.3.1 X-Ray

Source

The source of an X-Ray QL query consists of an alias name for a repository (see section 2.3.3

about "Repository") and a name for a XML document (see next section) in this repository.

XRaySource

xmlAlias

docName

getDocument()

Figure 4 - XRaySource class

To retrieve the XML document, the XML document cache is first looked up, as the XML

document may be already used before. If no document was found, it is retrieved from the

meta database. This XML document specifies the XML elements that will be affected by the

X-Ray QL query.

2.3.2 XML

Document

The X-Ray architecture [OBE02] allows creating a mapping between a XML DTD and a

relational database schema (also called repository). By storing this mapping information into

the meta database, it is possible to manage the relational data in XML documents.

In fact a mapping between one XML DTD and one relational database schema would result in

only one XML document covering all the data of the relational database schema (section 3.5

in [KIM02]). However, as the X-Ray query language allows operating on different XML

documents valid to a single DTD, it is necessary to use a mechanism for differentiation of

XML documents.

The mapping rules of X-Ray demand, that the DTD elements of the first level (DTD's root

element is level 0) are mapped to a database relation (see mapping rules in [KAP00] and

[ART99]). As the rows of every database relation are identified by primary key attributes, it is

possible to differentiate XML documents by the values of their first level elements. So it is

possible to manage more than one XML document for a mapping between a XML DTD and a

relational database schema.

X-Ray Architecture

Page 17

To manage the different XML documents they are stored in the XMLD

OCUMENT

relation in

the meta database.

XMLD

OCUMENT

@MappingId

@XMLDocumentName @XMLCP

@DBAttId @Value

@Key

ALL

author

europe

author

author1

1 europe

author

author2

1 germany

author

author1

Table 1 -

XMLD

OCUMENT

database relation

The XMLD

OCUMENT

AME

column stores the name of the XML document. The column

MappingId contains the mapping information that is used as repository of this XML

document.

A XML document consists of a first level XML content particle, where the XMLCP column

contains the name of this first level element. As a first level XML content particle has to have

a mapping to a database relation, it is possible to identify this relation by its primary keys

(DBattId). By providing values (Value) for every primary key attribute, a XML document

may be uniquely identified. As there may be more then one occurrence of a first level XML

content particle belonging to a document (the document "europe" contains the authors

identified by a database attribute with the value "author1" and "author2" for the primary key

attribute with id 1), the occurrences are separated by a key (XMLElementKey object). So the

Key column in the XMLD

OCUMENT

relation represents a set of primary key attributes,

belonging to a concrete first level element, or the other way round, all primary key attributes

(DocAttribute object) of a single first level XML content particle occurrence, with their

values, belong to a certain key.

The following figure shows the UML [UML01] class diagram for XML documents in X-Ray:

XMLDocument

docName

mappingID

xmlAlias

addElement()

getElements()

DocAttri

bute

attId

value

XMLElementKey

key

xmlCP

addAttribute()

getEttributes()

1..n

Figure 5 UML XML document class structure

X-Ray Architecture

Page 18

The XMLDocument object is a representation of a document stored within the

XMLD

OCUMENT

relation in the meta database. This object provides a function to add new

first level elements to the document, and to retrieve all first level elements from the

document. The first level elements are represented by the XMLElementKey class, which

contains all primary key attributes and according values of the first level element. The

DocAttribute class represents such an attribute.

When a new X-Ray repository is created by building a mapping between a DTD and a

relational database schema with the X-Ray init tool [OBE02], the XMLD

OCUMENT

relation

has to be initialised with the "Default XML Document". This document contains all the data

of the relational database schema. The database relation would look like this.

XMLD

OCUMENT

@MappingId

@XMLDocumentName @XMLCP @DBAttId @Value

@Key

ALL

author

Table 2 ALL document in

XMLD

OCUMENT

database relation

The value "*" indicates, that all occurrences of the primary key attribute belong to this XML

document. This "ALL" document is used, when an X-Ray QL query with an X-Ray source

without a XML document name is processed. The QL engine is responsible for administer

this meta database relation, by adding or removing XML documents.

2.3.3 Repository

An X-Ray repository is a certain mapping between a XML DTD and a relational database

schema. The mapping describes the assignments between XML elements of the DTD and

relational components. So a XML atomic type may be mapped to a database attribute (see

mapping rules in [KAP00] and [ART99]), which means, that the values for the XML atomic

element type in a XML document will be retrieved from this database attribute.

To identify the mapping information, the expression XML alias was introduced.

A repository can be divided in several XML documents, where the different documents may

overlap each other.

X-Ray Architecture

Page 19

Figure 6 - X-Ray repository LITERATURE

The graphic above shows the repository literature. This repository consists of the mapping

between the DTD LITERATURE_HISTORY.DTD and the relational database schema

containing data for some authors of the worlds literature history. There may exist manifold

mappings between XML concepts and relational concepts, as long as the X-Ray mapping

rules ([KAP00] and [ART99]) are fulfilled. According to the X-Ray mapping rules, the first

level element has to be mapped to a database relation (ET_R_direct). So the first level

element author is mapped to the database relation A

UTHOR

. The atomic element type

fname is mapped to the database attribute FNAME and the atomic element type lname is

mapped to the column LNAME.

The database relation the first level element is mapped to is used for differentiating XML

documents. In the graphic above the accommodation with ID value "author1" belongs to the

XML document "germany.XML". The XML document "europe.XML" contains both author

entries which overlaps with the first XML document, as mentioned above.

Additionally a universal "ALL" document exists, which contains all entries of the database

relation mapped by the first level element. The name "ALL" is therefore a reserved document

name and may not be used by regular XML documents. An X-Ray QL query that operates on

the "ALL" XML document would use the data, contained in the relational database, in a data

oriented way and not in document oriented way.

X-Ray Architecture

Page 20

Repository

dbSchema

mappingId

xmlAlias

xmlD TD

Repository()

getXMLDTD()

Figure 7 - Repository class

The X-Ray runtime system uses a Repository object and a repository cache to manage X-Ray

repositories. Every time the QL engine processes a X-Ray QL query the document's source, is

analysed, and a repository is searched for the XML alias, which is contained within the

query's source. As the repository consists of mapping information, which is not going to

change on the fly, the used repositories can be kept in a cache for faster access. So first the

repository is searched in the repository cache.

If no repository were found for the case insensitive XML alias, the repository would be taken

from the meta database. Therefore the meta database relations XMLA

LIAS

and

XMLDBS

CHEMA

APPING

are used to create a Repository object.

XMLA

LIAS

@XMLAlias MappingId

Status

literature

Final

Table 3 -

XMLA

LIAS

database relation

XMLDBS

CHEMA

APPING

@MappingId XMLDTD DBSchema

literature

Litdb

Table 4 -

XMLDBS

CHEMA

APPING

database relation

The constructor of the Repository class uses the generators of the X-Ray definition part to

retrieve the XML DTD (line 10 in code below) and the database schema (see line 12) for the

mapping at hand. The name of the XML DTD and the database schema for the available

mapping can be found in the XMLDBS

CHEMA

APPING

meta database relation. The mapping

is identified by the mapping id, but for comfortable usage the XML alias name was

introduced. The correlation between XML alias name and mapping id can be resolved with

X-Ray Architecture

Page 21

the XMLA

LIAS

meta database relation.

After generating the XML DTD and the database schema, the DTD is mapped to the database

schema using the mapping generator (see line18) of the definition part. With this initialisation

of the Repository object, the X-Ray runtime system is able to access the mapping

information at any time.

For quicker reuse the Repository object is placed in the repository cache.

/**

* This constructor creates the repository with the given parameters. The

* constructed Repository contains the XMLDTD mapped to the DBSchema.

* @param xmlAlias The alias for the mappingId of this repository.

* @param mappingId The mappingId for the mapping of XMLDTD and DBSchema.

* @param dbSchema The name of the DBSchema of this repository.

public Repository(String xmlAlias, int mappingId, String dbSchema) {

this.xmlAlias = xmlAlias;

this.mappingId = mappingId;

this.dbSchema = dbSchema;

try {

ConnectInfo conInfo = EnvironmentProperties.getMetaDBConnectInfo();

Connection con = conInfo.createConnection();

//use threads for mapping XMLDTD and DBSchema

//Thread1 retrieves XMLDTD with the given xmlAlias

10: ifs.xray.Thread1 t1 = new ifs.xray.Thread1(xmlAlias, con);

11: //Thread2 retrieves DBSchema with the given

12: ifs.xray.Thread2 t2 = new ifs.xray.Thread2(dbSchema, con);

13: try {

14: t1.run();

15: t2.run();

16: xmlDtd = t1.getXMLDTD();

17: DBSchema schema = t2.getDBSchema();

18: MappingEngine.map(mappingId, xmlDtd, schema, con);

19: }

20: catch (Exception ex) {

21: err.println(clsName, ex.getMessage());

22: }

23: }

24: catch (ConnectException ex) {

25: err.println(clsName, ex.getMessage());

26: }

27: }

Example 1 - Constructor of class Repository

An X-Ray repository can be identified by the XML alias, which must be unique in the X-Ray

meta database. For example the alias acc identifies the repository with the mapping

information between the XML DTD accommodation and the database schema with the

according database relations.

X-Ray Architecture

Page 22

A concrete XML document inside a repository is referred in the following way:

Example 2 - X-Ray Source structure

For example: literature:germany

This references the XML document "germany" in the repository "literature". So the document

is referred without the extension ".XML", as the name is always referring a XML document.

If no document name is specified, the "ALL" document will be used for data oriented

processing of the X-Ray QL query.

Chapter 3

X-Ray Runtime

Implementation

This chapter deals with the implementation of the X-Ray runtime part. The runtime system is

realised as a class library and implemented in Java. The implementation work was realised

with the integrated development environment (IDE) JBuilder in the version 5 using Java

SDK.

In the following sections, the components of the runtime part, mentioned in the last chapter,

will be described in detail.

3.1 Authorisation

3.1.1 Motivation

As not everyone is allowed to use the X-Ray runtime system, a user has to be authorised to

use the X-Ray QL runtime system. Therefore an authorisation mechanism is implemented

within the authorisation engine and is responsible for identifying the user. If the authorisation

engine recognised the user he will retrieve a unique session id, which he has to use for

identifying his session and indirectly himself, every time he is using the X-Ray runtime

system functionality.

3.1.2 Authorisation

Engine

If the user authorises with the authorisation engine, his identity will be verified. The users

identity consists of his login name, this is his username, and his login password, which is

represented by the Authorisation object.

X-Ray Runtime Implementation

Page 24

Authorisation

username

pass word

Figure 8 - Authorisation class

The authorisation engine takes this object and verifies the users identity. Therefore the

authorisation engine simply checks if the user's username and password are valid by trying to

find a user with this username-password combination. As this authorisation mechanism

controls access to the runtime system and the data contained in the meta database, the user

information is kept within the meta database. So the meta database was extended by a U

SERS

relation.

SERS

@UserName @Password

xrayuser

Userpass

Table 5 - Database relation users

This relation contains two attributes, the name and password used for authorising the user

with the runtime system. This relation might contain a lot more information about the user,

like first name, last name and so on. This authorisation mechanism permits to manage more

than one user by simply adding a new user to this meta database relation.

If the authorisation engine is trying to verify a user's identity, it will try to find a user object,

with the corresponding username and password.

User

username

password

Figure 9 - User class

Therefore the user cache is searched first, as the user cache would contain user objects that

have been used before. If no user object was found, the U

SER

relation within the meta

database would be searched for the combination of the given username and password. If a

user were found, a new user object would be created and would be put into the user cache. So,

if no user object can be retrieved, neither from the user cache nor from the meta database, the

user is unknown to the X-Ray runtime system, and therefore the authorisation engine will

X-Ray Runtime Implementation

Page 25

reject him. In this case no session id is created for the unknown user and the user will not be

able to execute X-Ray QL queries with the X-Ray runtime system.

If a user object were found, either in the user cache or in the meta database, the user would be

known to the runtime system. In this case the authorisation engine requests the session

management to create a new session for the user, which is identified by a session id. This

unique session identifier enables the user to invoke the QL engines functionality. In this way

the authorisation engine activates the session management of the X-Ray runtime system.

client

: QLEngine

: AuthorisationEngine

: MetaDBAccess

: SessionManagement

: UserCache

authorise(Authorisation)

createSession(String)

If a user could be

ass ociated with

the Authorisation

object

getUser(String, String)

getUser(String)

Figure 10 - Sequence diagram of the authorisation mechanism

3.2 Session

Management

The session management is responsible for creating a session object, when the authorisation

engine authorises a user. In this case a new user session will be created and will be put in the

session cache.

The user's session is identified by a unique session ID, which consists of ten characters,

generated accidentally, and a uniquely rising number. The user has to specify this session id

every time he requests functionality from the X-Ray runtime system. With the provided

session id the session management allocates his session by taking the according session object

from the session cache and initialising the runtime system with the user's information.

In this way the session management enables different users to use the X-Ray runtime system

at the same time, without re-authorising at the beginning of every request. To accomplish this

comfort, on the one hand the connections to the relational databases used by a user are stored

within this user's session object. On the other hand all uncommitted insert operations and the

according SQL query to insert the XML document in the meta database are stored in this

user's session object. This is because the X-Ray QL insert queries are executed during a

"COMMIT" operation and not during the "INSERT" operation, as not all relational data may

X-Ray Runtime Implementation

Page 26

be available with one X-Ray QL insert query (see section 3.10). Therefore the insert

operations and the XML document that shall be inserted are also kept in the session object.

Connection to a relatonal

Database identified by a

ConnectionInfo object.

Connection

InsertQuery

Inserts are kept together

according to the

XMLDocument the

belong to

Session

username

sessionID

connections

inserts

XMLDocument

documents

Figure 11 - Session management class diagram

As it is possible to use different repositories (see section 2.3.3) as source of different X-Ray

QL queries, there may be connections to different relational databases. If a new source (see

section 2.3.1) is used in an X-Ray QL query a new relational database connection will be

created. Every relational database connection in a session is created in non-auto-commit

mode, this means, that a commit or rollback has to be invoked explicitly, to end a transaction.

As mentioned before, a session also holds all insert queries the user sends to the X-Ray

runtime part. As several XML documents may be created in parallel, the session has to store

insert queries separated by their source. If an insert query is executed during a commit

operation (see section 3.12.1), it will be removed from the session object, so it cannot be

executed a second time. Therefore the session object provides functionality to add, remove,

retrieve and update insert operations within the session.

The XML documents (see section 2.3.2), which shall be created, are also kept within the

session. The documents will remain in the session until a valid commit operation inserts them

into the meta database. Providing these information it is possible to spread transactions over

several requests without loosing connections to the used relational databases. So it is not

necessary to commit or rollback any request sent to the runtime system. An example will

clarify the session management functionality:

Request 1, User 1:

INSERT ELEMENT lithist FROM literature:england;

INSERT ELEMENT author WITH ATTRIBUTES @id="author3", @firstentry="03.10.2002",

@lastentry="03.10.2002" INTO lithist FROM literature:england;

Example 3 - Request 1, User 1

Request 1, User 2:

X-Ray Runtime Implementation

Page 27

SELECT lithist FROM literature;

Example 4 - Request 1, User 2

Request 2, User 1:

INSERT ELEMENT name INTO /author[@id="author3"] FROM literature:england;

...

COMMIT;

Example 5 - Request 2, User 1

Two time-shared users send X-Ray QL queries to the X-Ray runtime system. For every user a

session object will be created. In the first request of user 1 a new XML document with the

name "england" is created by inserting two elements into the not existing XML document

"england" of the repository "literature". A repository is a mapping between a XML DTD and

a relational database, and is described in section 2.3.3 "Repository". As user 1 has to provide

his session id to execute an X-Ray QL query with the X-Ray runtime system, the insert

statements and the new XML document can be added to the user's session object. In the first

request of user 2, all lithist elements are selected, however, as the two insert operations

of user 1 have not been committed yet, the new XML document "england" may not be

selected yet. In the second request of user 1 a third element is inserted into the new XML

document. The according X-Ray QL insert query is again added to the user's session object.

At the end of the second request, all X-Ray QL queries contained in the session object of user

1 are committed by a "COMMIT" operation and the XML document is inserted into the meta

database.

3.3 QL

Engine

The top-level component of the X-Ray runtime system is the X-Ray QL engine.

authorise

create session

execute query

verify query

check sy ntax

process query

process query if

query has to be

executed

free session

X-Ray user

log out

Figure 12 - Use case diagram for QL engine tasks

X-Ray Runtime Implementation

Page 28

This engine is the direct interface for user access. It accepts X-Ray QL queries and transmits

the query result after, processing the query, to the user. For this reason it provides

functionality to verify or execute X-Ray QL queries.

QLEngine

authorise(auth : Authorisation)

verify(query : String, sessionID : String)

execute(query : String, sessionID : String)

freeSession(sessionId : String)

Figure 13 - QLEngine class

As the verify and execute methods are only available to authorised users the user interface

provides an authorisation method. The authorisation of a user is simply delegated to the

authorisation engine (section 3.1), which verifies the users identity and creates a session for

an authorised user. This session may be removed by the freeSession method, which will be

called when the user stops communicating with the X-Ray runtime system interface.

The verify method verifies the syntax of an X-Ray QL query and returns the result of the

verification. In case of an incorrect syntax the number of errors and the according syntax

errors are returned.

The execute method executes an X-Ray QL query and returns the result of the execution.

This may be a list of syntax or semantic errors, in case of a failure during execution, or a

result string indicating the success of the execution.

Several engines support the QL engine in executing or verifying an X-Ray QL query, so that

the QL engine only needs to organise the supporting engines. In the following sections the

execute and the verify method are documented in detail.

3.3.1 X-Ray QL Query Execution

When the user initiates the execution of a number of X-Ray QL queries, the QL engine first

checks the provided session id by retrieving the according session object (see line 13 in code

extract below). If a session object was found for the session id the QL engine starts the

execution process of the query (see sequence diagram below).

X-Ray Runtime Implementation

Page 29

: QLEngine

: Scanner

: Parser

sqlEngine :

SQLEngine

: XPathEngine

init( )

parse( )

init( )

execute( )

init(String, XMLDTD, SQLEngine)

execute( )

Figure 14 - Sequence diagram for executing an X-Ray QL query

The first step of the QL engine execution process retrieves single X-Ray QL queries from the

query stream. Therefore the QL engine initialises the syntax scanner with the string

containing the queries (line 20). Afterwards, the syntax parser is required to parse the queries

and separate them into single queries (line 21). The resulting queries are stored in a sorted

Map. If syntax errors occur while parsing the queries, the execution will stop at this point

(line 27).

Every single query is processed separately in order of appearance (processQuery method on

line 37). Thereby for every X-Ray QL query, of type other than transaction operations, the QL

engine creates a SQL engine instance according to the type of X-Ray QL query. So if the

currently processed X-Ray QL query is an insert operation, an insert engine (sub class of

SQLEngine) will be created, if the query is an update operation, an update engine (sub class

of SQLEngine) is created, and so on. Then the QL engine initialises the XPath engine with a

concrete SQL engine instance (e.g. InsertEngine), and starts the XPath engine.

On the one hand the XPath engine uses the XPath expression of the X-Ray QL query to

retrieve a target XML element. This is a XML element of the DTD, which belongs to the

XML document of the query's repository. On the other hand all predicates contained in the

query are extracted. Both, target element and predicates, are fed into the SQL engine instance.

As a last step of execution the SQL engine instance is executed and the result of the X-Ray

X-Ray Runtime Implementation

Page 30

QL query is retrieved and returned to the QL engine.

As executing insert operations or executing queries on before unused relational database

connections could have modified the session object, the session has to be updated (see line

45).

At the end of the execute method, the execution result of the queries in the query string is

returned to the user.

/**

* Executes a statement.

* @param query statement to verify

* @param auth authorisation of the user who wants to execute this method

* @return Result of execution

7: */

public static String execute(String queryStr, String sessionId) {

ifs.xrayruntime.engine.syntax.ErrorStream error = new

ifs.xrayruntime.engine.syntax.ErrorStream();

10:

QueryResult res = new QueryResult();

11:

//verify the users session id

12:

try {

13:

session = SessionManagement.getSession(sessionId);

14:

}

15:

catch (SessionManagementException ex) {

16:

res.addQueryOutput("Error: "+ex.getMessage());

17:

return res.toString();

18:

}

19:

try {

20: Scanner.Init(queryStr, error);

21:

Map queries = Parser.Parse();

22:

if (error.count != 0) {

23:

Enumeration enum = error.getErrors().elements();

24:

while (enum.hasMoreElements())

25:

res.addQueryOutput((String)enum.nextElement());

26:

res.addQueryOutput(error.count + " error(s) detected");

27:

return res.toString();

28:

}

29:

//init SQL Engine

30:

SQLEngine.init(session);

31:

//process queries

32:

Iterator it = queries.values().iterator();

33:

while (it.hasNext()) {

34:

XRayQLQuery query = (XRayQLQuery)it.next();

35:

res.addQueryOutput(query.toString());

36:

try {

37:

res.addQueryOutput(processQuery(query).toString());

38:

}

39:

catch (XRayQLException ex) {

40:

res.addQueryOutput("Error: while executing query: "+ex.getMessage());

41: return res.toString();

42:

}

43:

}

44:

//update used session

45:

SessionManagement.updateSession(session);

46:

}

47:

catch (Exception ex) {

48:

err.println(clsName+".execute()", "Cannot execute query: "+queryStr+"

("+ex.getMessage()+")");

X-Ray Runtime Implementation

Page 31

49: res.addQueryOutput("Cannot execute query:

"+queryStr+"("+ex.getMessage()+")");

50:

}

51:

return res.toString();

52: }

Example 6 - Execute method of QLEngine

3.3.2 X-Ray QL Query Verification

The verification of an X-Ray QL query consists only of a syntax check of the queries.

Therefore the QL engine initialises the syntax scanner with the query string containing the X-

Ray QL queries and tells the syntax parser to parse these queries. If the syntax were not valid,

the syntax errors would be returned to the user.

: QLEngine

: Scanner

: Parser

init( )

parse( )

Figure 15 - Sequence diagram of a X-Ray QL query verification

3.4 Syntax

Check

Scanning the syntax of an X-Ray QL query is essential, because it would be senseless to start

executing a query operation, if the syntax was incorrect. Therefore the syntax scanner and

parser check the validity of the query syntax.

The syntax scanner is responsible for scanning the character stream of the query. The syntax

parser verifies the correctness of symbols and their correct order. If the syntax verification

failed, error messages would be written to an error stream. These three components have been

generated by the compiler-compiler CoCo/R for Java [MÖS90], a tool developed at the SSW

department of professor Mössenböck. The compiler-compiler generates among other classes a

parser and a scanner with the use of an input language (LL(1) [ALT01]). The X-Ray query

X-Ray Runtime Implementation

Page 32

language was developed to fulfil this characteristic.

Parser

parse()

ErrorStream

getErrors()

Scanner

init()

Figure 16 - Syntax engine class diagram

The following example gives a rough impression of the input language. It shows the top-level

grammar rule of X-Ray QL (XRayQL) that contains references to the Operation grammar

rule.

XRayQL <^Map queries>

(. queries = new TreeMap();

XRayQLQuery query;

int i = 1; .)

Operation

<^query>

";"

(. queries.put(new Integer(i++), query); .)

{

Operation

<^query>

";"

(. queries.put(new Integer(i++), query); .)

Example 7 - X-Ray QL augmented grammar

The core of this specification is the EBNF rule

XRayQL = Operation ';' {Operation ';'}.

Example 8 - X-Ray QL core EBNF rule

It is augmented by attributes and semantic actions. The attributes (e.g. <^query>) specify the

parameters of the symbols. There are input attributes (e.g. <x, y>) and output attributes (e.g.

<^z>). Output attributes are preceded by a "^". A grammar symbol may have at most one

output attribute, which must be the first attribute in the attribute list (e.g. sym<^x, y, z>). The

semantic actions are arbitrary Java statements between "(." and ".)". The generated parser at

its position in the grammar executes a semantic action. The XRayQL rule declares that the X-

Ray query language consists of at least one operation. If there is more than one operation, the

following operations are separated by a semi-colon. The Operation rule defines the permitted

X-Ray QL operations. These are the select, delete, rename, insert, update and transaction

Details

Seiten
Erscheinungsform: Originalausgabe
Jahr: 2002
ISBN (eBook): 9783832475802
ISBN (Paperback): 9783838675800
DOI: 10.3239/9783832475802
Dateigröße: 895 KB
Sprache: Englisch
Institution / Hochschule: Johannes Kepler Universität Linz – Informatik, Angewandte Informatik
Erscheinungsdatum: 2004 (Januar)
Note: 1,0
Schlagworte: relationale datenbank meta abbildung datenmanipulationssprache

Autor

Christian Hiebl (Autor:in)

Implementation of a declarative query and data manipulation language for X-Ray

Zusammenfassung

Leseprobe

Inhaltsverzeichnis

Details

Autor

Christian Hiebl (Autor:in)