structWSF is based on what is known as a Web-oriented architecture (WOA) , which can be defined as:
WOA is an architectural foundation of the Web, and is a subset of the service-oriented architectural (SOA) style, wherein discrete functions are packaged into modular and shareable elements ("services") that are made available in a distributed and loosely coupled manner. WOA uses the representational state transfer (REST) architectural style defined by Roy Fielding in his 2000 doctoral thesis; Fielding is also one of the principal authors of the Hypertext Transfer Protocol (HTTP) specification.
REST provides principles for how resources are defined and used and addressed with simple interfaces without additional messaging layers such as SOAP or RPC. The principles are couched within the framework of a generalized architectural style and are not limited to the Web, though they are a foundation to it.
REST and WOA stand in contrast to earlier Web service styles that are often known by the WS-* acronym (such as WSDL, etc.). WOA has proven itself to be highly scalable and robust for decentralized users since all messages and interactions are self-contained (convey "state").
structWSF abstracts its WOA services into simple and compound ones (which are combinations of the simple). All Web services (WS) have uniform interfaces and conventions and share the error codes and standard functions of HTTP. We further extend the WOA definition and scope to include linked data, which is also RESTful. Thus, our WOA also sits atop an RDF (Resource Description Framework) database ("triple store") and full-text search engine.
These Web services then become the middleware interaction layer for general access and querying ("endpoints") and for tying in external software ("clients"), portals or content management systems (CMS). This design provides maximum flexibility, extensibility and substitutability of components.
As you can notice in the schema below, the general architecture of this system is split in four main areas:
- The structWSF Web Services Framework
- CMS interacting with the structWSF
- External Users interacting with the structWSF
- External datastore systems used by the structWSF.
The Web Services Framework Middleware
The core to the system is the Web services middleware layer, or structWSF. This Web services framework (WSF) is the abstraction layer that provides the Web service endpoints and services for external use. It also provides the direct hooks into the underlying RDF triple stores and full-text search engines that drive these services. At initial release, these pre-configured hooks are to the Virtuoso RDF triple store (via ODBC, and later HTTP) and the Solr faceted text search engine (via HTTP). However, the design also allows other systems to be substituted if desired or for other specialized systems to be included (such as an analysis or advanced inference engine).
The controlling Web service in structWSF is the Auth: Validator Web service (see below). The initial version uses registered IP addresses as the basis to grant access and privileges to datasets and functional Web services. Later versions may be expanded to include other authentication methods such as OpenID, keys (à la Amazon EC2), foaf+ssl or oauth. A secure channel (HTTPS, SSH) could also be included.
Other Core Web Services
The other core Web services provided with structWSF are the CRUD functional services (create - read - update - delete), import and export, browse and search, and a basic templating system. These are viewed as core services for any structured dataset.
In initial release, the import and export formats include RDF/XML, RDF/N3, RDF/Turtle and XML.
Datasets, Access and Use Rights
A simple but elegant system guides access and use rights. First, every Web service is characterized as to whether it supports one or more of the CRUD actions. Second, each user is characterized as to whether they first have access rights to a dataset and, if they do, which of the CRUD permissions they have. We can thus characterize the access and use protocol simply as A + CRUD.
Thereafter, a simple mapping of dataset access and CRUD rights determines whether a user sees a given dataset and what Web services ("tools") are presented to them and how they might manipulate that data. When expressed in standard user interfaces this leads to a simple contextual display of datasets and tools.
At the Web service layer, these access values are set parametrically. The system, however, is designed to more often be driven by user and group management at the CMS level via a lightweight plug-in or module layer.
A Structured Data Foundation
Fundamentally, this "data-driven application" works because of its structured data foundation. structWSF employs an innovative design that exposes all RDF and record aspects to full-text search and is able to present contextual ("faceted") results at all times in the interface . In addition, the Virtuoso universal server provides RDF triple store and related structured data services.
The actual "driver" for the structured data system is one or more schema ("ontologies") setting all of these structured data conditions. These ontologies are also managed by the triple store. The definition of these ontologies is specified in such a way with accompanying documentation to enable new scopes or domains to easily drive the system.
Interactions with CMSs and External Clients
As so far described by the diagram above, all interactions with the system have been mediated either by Web service APIs or external endpoints, such as SPARQL.
For external clients or any HTTP-accessible system, this is sufficient. Programmatically, external clients (software) may readily interact with each structWSF Web service and obtain results via parametric requests.
However, the framework is also designed to be embedded within existing content management systems (CMSs). For this purpose, an additional layer is provided.
The architecture of the system can support interactions with standard open source CMSs or app frameworks such as Django, Drupal, Joomla!, Liferay, Ruby on Rails, or WordPress, as examples. CMS interactions first occur via specific modules or plug-ins written for that system. These are very lightweight wrappers that conform to the registry and hooks of the host CMS system. The actual modules or plug-ins provided are also geared to the management style of the governing CMS and what it offers. Each module or plug-in wrapper is a packaging decision of how to bound the structWSF Web services in a configuration appropriate to the parent CMS.
This design keeps the actual tie-ins to the CMS as a very thin wrapper layer, which can embrace an open source licensing basis consistent with the host CMS. Because all of the underlying functionality has been abstracted in the structWSF framework, licensing integrity across all deployment layers is maintained while allowing broad CMS interoperability. The design also allows networks to be established of multiple portals or nodes with different CMSs, perfect for broad-scale collaboration.
In initial release, structWSF has been linked to the Drupal CMS as the conStruct distribution. conStruct SCS uses the contributed Drupal Organic Group module to manage user and dataset permissions.
The Central Auth: Validator Web Service
The core of structWSF is the Auth: Validator Web service. This service validates all queries sent to any Web service endpoint of within structWSF.
The CMS is intended to use some of the Web services endpoints to manage users subscriptions and such. Otherwise, all users are expected to use the Web services endpoints oriented to the user.
Benefits of the structWSF Design
This design has some key benefits:
- Broad suitability to a diversity of deployment architectures, operating systems and scales
- Substitutability of underlying triple stores and text engines
- Substitutability of preferred content management systems
- Access and use of Web service endpoints independent of CMS (external clients)
- Performant Web-oriented architecture (WOA) design
- Common, RESTful interface for all Web services and functions in the framework
- Easy registration of new Web services, inclusion with authorization system
- Ability to share and interoperate data independent of client CMSs or portals
- Use of the common lingua franca of RDF and its general advantages .
- ↑ See http://www.mkbergman.com/?cat=153
- ↑ Frédérick Giasson, 2009. RDF Aggregates and Full Text Search on Steroids with Solr, April 29, 2009. See http://fgiasson.com/blog/index.php/2009/04/29/rdf-aggregates-and-full-text-search-on-steroids-with-solr/
- ↑ Michael K. Bergman, 2009. Advantages and Myths of RDF, white paper from Structured Dynamics LLC, April 22, 2009, 13 pp. See http://www.mkbergman.com/wp-content/themes/ai3/files/2009Posts/Advantages_Myths_RDF_090422.pdf