The World Wide Web (WWW) is a huge, distributed information store. It has been very successful in providing information to users all over the world. The development of protocols such as the Common Gateway Interface (CGI), and technologies such as proxy servers and gateways, have enabled construction of numerous web-based applications, such as collaborative editors and annotation systems.
However, the information retrieved from the web is presented to users on an as-is basis without a "big picture" that associates the structure of the data with the specific application. In other words, users always view the web according to the low-level structure determined by hyperlinks within the web pages that contain the actual information.
A Web View is an abstract structure imposed on the web, rather than defined entirely within the web pages themselves. Different users might have different views of the same subweb; the same user might have different views at different times, appropriate to the task at hand.
We show through some motivating examples that dynamically customizable web views would be very useful. We then present an architecture and an implementation that integrates OODB technology with the web to support views that can be dynamically customized for web-based application systems.
Say one physicist wants to use the experimental data posted on the web by another physicist, but in some different way. That is, the data is hyperlinked in a structure that matches the author's understanding of the logical relationships but the reader's perspective is different. The reader should then create a new view over the data to restructure and/or reformat the pages, for human browsing and/or input to computational tools.
Some healthcare systems use WWW to store patient information such as personal data, diagnoses, healthcare history, test results, physician notes, etc. Patients may be admitted to different hospitals within even a short period concerned with a single illness or injury, so logically centralized information may be distributed over multiple websites. Patient data should not be identically visible to all (authorized) users: administrative personnel don't need to see CT images, whereas medical researchers should mine histories and clinical trials anonymously. Thus information should be structured differently for each user role and/or application, preferably dynamically to maximize flexibility as new treatments or studies are considered.
In a software development environment that uses WWW for storage of design documents and source code, the designers and programmers would like to view the software system under development in different ways. The designers should see the system primarily as chapters and sections of design documents, perhaps with hyperlinks to corresponding code. Programmers, on the other hand, should see the system in terms of subsystems, modules and source files, perhaps with hyperlinks to relevant fragments of the design. These two sets of objects may be logically related in hierarchical structures illustrated in Figure 1.
Finally, suppose the web is used in a training system where an experienced user sets up a "trail" of web visits and operations for novice users to exercise. The view then consists of a temporal series of visits to web pages and encapsulates the operations (input) of the "teacher". To minimize authoring effort, the view might be constructed dynamically by tracing the experienced user's web traversal.
The main way to provide customized views of the web is currently through "homepages" with hyperlinks to web pages of interest to the homepage author. A homepage may include queries to search engines (Alta Vista, Lycos, Yahoo, etc.), with limited tracking of web changes, but is relatively static with the hardcoded URLs generally modified by hand. Homepages alone cannot provide application-specific (and thus more powerful) query ability.
We identify some essential requirements for dynamically customizable web views:
The architecture presented in this paper uses Object Oriented Databases (OODB) to perform the core functionalities including view and meta-information storage and query processing. OODBs provide a suitable data model for WWW artifacts: web data is presented by attributed objects with referential and composite relationships, in addition to page content and embedded hyperlinks. Also, the rich set of associative and/or navigational functions provided by most OODBs enables powerful search capabilities. Our architecture does not require modifications to current Web protocols or browsers.
The system architecture design is the key issue in supporting dynamically customizable views on the web. Our system architecture is shown in Figure 2. It consists of three major components:
To support dynamically changing web data, the View Processor must be able to handle situations like unavailable web objects because of network failure.
In addition to the architecture, we also introduce the concept of embedded method of web objects. An embedded method is defined in the form of an "unknown tag" in HTML. The specification of HTML defines that "unknown tags" should be ignored by browsers, therefore it will not be shown via a normal access. But when the web object is accessed by the View Processor, the View Processor may understand the information carried by the embedded methods. The information can be an attribute value that describes characteristics of the web object or can be the definition of an update operation. For example:
<TITLE> Example of Embedded Methods </TITLE> <H1> Example of Embedded Methods </H1> <EMETH type=attribute name=author value=jyang@cs.columbia.edu> <EMETH system=DKWEB type=operation name=NewComment npara=2 para1=from para2=content method=GET operator=http://www.cs.columbia.edu/~jyang/cgi-bin/sendmail.cgi> ...
Embedded methods are meant to make the web pages Objects. Currently the web pages are mostly hyperlinked passive data (which is what they were initially designed for). In order to use the web as an OODB, we need them to have abilities to provide attribute values and operation methods like an object has.
In particular, Embedded method provides a way to define customized operation. For example, in the example above, the NewComment is implemented through sending an email to the author, alternatively, the cgi-bin can do different things such as appending the comment to the page automatically. The embedded methods should be written in some format that is expected by the View Processor. For example, the View Processor need to identify the word from and content and give them correct meanings (in this case, the person who gives the comment and the comment itself). Different View Processors may understand these methods (especially how to define parameters) differently, which is a problem when the web page is used by unexpected View Processors. Therefore, we propose to use the system keyword to distinguish different systems (such as in the NewComment example).
The system whose architecture is shown above goes through a number of main steps to construct a view.
Request Acceptance: The HTTP Implementation component receives requests from client browsers in the form of a GET or POST method. GET and POST are the two major methods (or commands) accepted by HTTP servers. They are used to distinguish different protocols used in the HTTP session. Usually GET methods are used to obtain a pre-defined view and POST methods are used to modify views. The HTTP implementation could translates the requests into OODB queries or update commands, and forwards the queries or updates to the OODB or the View Processor depending on the type of the request.
View Retrieval/Update: The OODB gets requests from the HTTP Implementation and either updates or retrieves view objects. If a request updates a view, the OODB informs the HTTP Implementation about the result and replies to the client browser and thus finish the operation. Otherwise, the next two steps are invoked.
View Construction: After the view object is retrieved from the database, it is sent to the View Processor, which parses the view definition and pulls in necessary data from either local database or from the web. The View Processor queries the objectbase for source location of the objects and either contacts local databases for application objects or the appropriate web site for remote web objects.
Reply: The View Processor then gives the processed view back to HTTP Implementation, and the processed view is ultimately sends back to the client browser.
Here we present three variants of the architecture described above. They vary from highly centralized model to highly distributed model. In this section we go over the different variants and compare them briefly.
The first variant is a highly centralized model. It uses a OODB to store virtually all the information about the web objects, including their meta-information (stored as attributes) and their location. The web objects are totally passive. They may provide information to the OODB by embedded methods but the OODB never get informed of their change. Therefore the OODB is very heavy loaded and has to reach out and search for the changes like what most of the web indexing search engines (e.g. Alta Vista) do. Figure 3 shows how this kind of system is formed:
The main advantage of this model is simplicity. It requires minimum
support from the web object owners but if a web object provides embedded
methods, it could still use it. The problem with this model is
salability. The objectbase might end up with managing a huge amount of web
objects and responsible for all of their changes.
The second variant is the opposite of the first one, it is a
highly distributed model where the OODB may be degenerated to minimum
and the whole process engine might just be a slightly enhanced proxy
server. All the information are stored in the web objects and the
user depend on them. Figure 4 shows a system
diagram.
The advantage of this model is salability where all the information is
online and need not to be duplicated in a great deal. On the other hand, it
depends on the web object owners to carefully program their web objects. This
is usually too much an assumption but might fit applications where the web
object owners are technique-oriented persons. An example of such applications
is a software development environment where the web object owners are
designers, programmers and quality assurance person-el.
The third variant is a hybrid of the above two. It uses OODB as in
variant one to store meta information about the web objects and benefits from
the web objects that are carefully programmed through embedded methods.
The architecture is shown in Figure 5.
In the example shown above, there are two independent OODBs operating on
the same subset of the web. They store meta-information of the web objects
and view objects in order to provide a rich set of facility independent of the
web objects. It also allows an object in the OODB to point to a set of
objects by maintaining a URL to a directory or a index page. This gives this
variant more scalability: one to one relationship between OODB objects and web
pages is too costly when the number of web pages becomes very large.
In addition to that, the web objects may have links, which we call
backlinks to the OODBs that contain their meta-information. The
backlinks could be implemented simply as URLs because the objects in
the OODBs are accessible via HTTP. The backlinks are useful for
propagating updates of the object among all the OODBs that share it. When
the object is accessed through one of the OODBs, the backlinks can be
automatically followed and the OODBs that references the web object can update
their meta-information.
The system architecture described above is relatively high level. This is
because the architecture itself is generic. Different applications might
choose different approaches in their implementations. In this section several
important issues are addressed and possible solutions are discussed.
When data on the web is referenced and meta-information about the data is
stored in the local objectbase, an availability problem arises. A certain web
object may not be available at any time because of problems such as network
failures. Moreover, the meta-information stored locally may be out-of-date
because the web objects could vary without notice. There are two possible
solutions to this, and we call them Variable data and Immutable data.
For the Immutable approach, the local database keeps a copy of the web
object. When a web object is referenced, the View processor queries the
objectbase for a local copy and uses it as the real data. This approach is
appropriate the applications such as the training systems described in the
Motivating Example section. However, the Immutable approach does not provide
user with most up-to-date information the web object contains.
For the Variable data approach, the meta-information stored in the
objectbase is basically the URL of the web object and the web object is
fetched every time it is referenced. This approach provides the most precise
information about the web object but there exists a possibility that the
meta-information becomes out-of-date. This is because when the actual web
object changed its content, the objectbase might not get informed (which is
the usual case), and therefore the meta-information reflects an older version
of the web object.
There is, however, a hybrid approach. A copy of the web object is stored
locally. When the object is queried, the View processor would try to access
the original copy while it queries the local database. If the
meta-information of a web object can be obtained from its content, this
approach would handle the availability problem and provide reasonably
up-to-data information. Since the two operations are in parallel, the
performance overhead wouldn't be a big problem. But there would be a lot of
redundant information in the local objectbase that occupies a lot of local
resources.
Active data (when the web objects change, they call up the objectbase and
modify their meta-information) is certainly another approach to this problem.
In the variant three of our architecture, the OODBs update each other when the
web object is references. It would have a lot of overhead for the object
owners and requires a standard for web object editing which is not available
(and not practical). It is useful when the web object is semi-local: it is
not in the objectbase but in the same system that provides customizable views.
In this case an editing tool can be provided and enforced so that whenever an
essential change is made, it informs the objectbase to update the
meta-information.
There are basically two ways to implement the architecture described
above: using Common Gateway Interface (CGI) or building a new HTTP server (aka
httpd).
In a CGI implementation, one uses a standard HTTP server and the CGI
protocol to implement database queries from the HTTP server. The system is
broken down into several small pieces of cgi-bins and each of them does
certain kinds of queries or updates. The advantage is the system is highly
flexible in the sense that changing one or more of the cgi-bins changes the
behaviour of the system easily. Also, commercial HTTP servers can be used so
that their special features (e.g., secure HTTP) are still available.
In a server implementation, standard HTTP servers are not used as the HTTP
Implementation. Instead, it is build to fit specific application. This
approach also has its advantages. This is useful when some essential features
of HTTP (such as PUT method) are not implemented by most of the servers.
Also, it can be designed to be more efficient for the specific application.
For example, when a simple request is received, it does not need to spawn
another operating system process or thread but just perform the request by
itself.
By "server" we also include the case where the system is implemented as
a proxy server. A proxy server is a server known by the client browser so
that every HTTP request goes to it and the proxy server goes out to fetch the
URL and replies to the browser. Using a proxy server has the unique advantage
of tracking clients who access web objects not in the objectbase easily.
Because each access goes through the proxy server, the proxy can modify the
web objects that are sent back to the browser. For example, a useful
modification may be adding an extra form at the end of the web object which
would allow a user to add a web object into objectbase on the fly. An
implementation using this technique is described briefly in the Implementation
section.
Generally, we advocate the CGI implementation which allows more
flexibility and doesn't have to catch up with HTTP protocol changes. For some
applications where the user's access track is important, proxy server
implementation is a good option.
In systems where annotations or co-authoring is of interest, web objects
may be the target of update actions. These objects are not stored in the
local objectbase but are often considered part of the system. To provide
flexible, usually user-oriented schema for updating these semi-local web
objects, embedded methods are very useful.
Embedded methods are modeled by the objectbase, probably provided
with default values (e.g. they would send email to the owner of the web object
when someone makes an annotation).
As an example, in a co-authoring system, the co-author might be allowed to
modify the original copy. In this case, the owner would put an embedded
method which is recognized by the View Processor (but not by a normal
browser) to indicate a new method for update. The simplest way to do this
would be to have a URL to a cgi-bin which takes the modified version and
updates the original copy according to it (version controls through systems
such as rcs can be done).
Using embedded methods to customize operations allowed on a web
object provides additional flexibility as an alternative of the PUT method.
Since the PUT method (which is designed to write web objects) is usually not
implemented by HTTP servers, therefore embedded methods would be necessary.
Another advantage of embedded methods is that the owner of the web page
can access the objectbase via the methods. In this case, the web object
owner includes queries or update commands in the method and the View processor
could perform these database operations when they are called. This provides a
space for a web object owner to design what operations are done (and how) to
the object according to the information stored in the objectbase.
In a system that provides dynamically customizable views, authentication
and security is an important issue. Because the views could be updated
dynamically, the system must make sure that they can be updated only by users
with certain permission. Furthermore, when views themselves are important
and sensitive information, read access might need to be controlled as well.
For example, in a trailing system where an artist sets up a traversal
route through galleries and provides comment to pieces of art works, the
artist does not want other people to access the route and comments thus might
want to require a registration so that only authenticated users (probably
those who paid a service fee) would have access to comments.
A lot of research has been done on this topic so that we don't intend to
provide some novel general solution. Basically if the HTTP communication
itself is secure and authenticated, the problem is already solved. Otherwise,
the system might have to use technologies like ``cookies'' to enable the
server remember which user has what kind of access permission.
There are many possible ways to implement the architecture described
above. For the highly distributed model (or our variant 2), changing
browsers, either totally or through some client-side customization (e.g. CCI),
is possible. However, as we have analyzed in the Requirement section: using a
new browser limits inter-operability.
On the server side, there are CGI and server implementations which we have
already seen above. CGI implementation has the advantage of flexibility
while a server implementation provides potentially higher efficiency.
In between, we have the proxy server implementation has almost all the
advantages of a server implementation but since browsers can have only one
proxy server, this limits how many systems the users can access.
Java provides another possibilities: one can implement specific clients.
The client can connect to an OODB to get information and construct views.
This approach provides flexible operations on view construction (i.e., the
user can help the View Processor, which is written in Java, interactively when
it constructs the views). However, it is not always necessary to have such
flexibilities and it has performance penalty (e.g. transferring Java applets
introduces more network delay, executing the Java is slower than executing
compiled code at server side) so the implementation should use this technology
only when necessary.
We developed a sample realization of the architecture discussed above in
the Programming System Laboratory
at Columbia University . In our
implementation, we use an Object Oriented Database called Darkover,
which is also developed locally in our lab.
The sample system supports a shared web navigation and annotating system.
It provides the following features:
The implementation is illustrated in Figure
6. The implementation is CGI based, where there are several major
CGI scripts that perform the main functionalities of the system and
the Darkover objectbase supports the object-oriented data
models and various kinds of queries.
Darkover is an Object Management System with a rich set of object
definition and manipulation facilities: it supports a wide variety of
primitive data types (e.g. integer, string, enumerated) as well as
user-defined object classes; it supports both compositional hierarchy and
referential relationships; also a rich set of operations including
modification of attributes and objects are provided.
The View Processor contains a lot of CGI scripts. get_view
queries Darkover to find a view. In a view, each web object is shown
as a hyper link with some of the meta-information (e.g. last visited, size,
etc). When the user wants to receive the web object he/she follows the
hyperlink and the link is a reference to get_web_obj.
get_web_obj first looks at the objectbase to see if it is a local
object, and fetches from the original site it is not. After the real web
object is fetched, the script also parses the page and modifies all the
references so that further references are re-directed to this script. This
insures the tracking of the browser access to all the web objects.
get_web_objalso adds three forms at the end of the page so that
user can:
edit_web_obj is called when the edit form at the bottom of a web
object is submitted. it checks if the web object defines it's own update
method, if the update method is defined, the edited source is given to the
method as one of its parameters. Otherwise, the modified version is stored in
the objectbase. Later on, when other references to this web page is made, the
modified version is attached to the end of the original page as a hyper link
so that other users can see the modification as well.
Comments are stored in the objectbase as composite objects of the web
object. Also, if the web object defines a method named NewComment,
all the comments to it will be forwarded to the owner by that method.
A related project by Steve Dossick (need reference number here) in the
Programming Systems Lab of Columbia University embodies the concept of
dynamically customizable view. His system is to interface a multi-user
Process Centered Software Development Environment (PCE) through the web.
Its View Processor is implemented as a proxy server. Every browser access
goes through the proxy server, if it is a request concerning the PCE then it
is forwarded to the environment server. The web objects are implemented in a
Immutable approach where all the data are physically resident in the
environment objectbase of Oz.
The stress of OreO is in "constructing highly specialized
transducers that can be composed to produce more sophisticated
aggregate behavior". The aggregate behavior, however, does not
address the dynamic property of the architecture described in this
paper.
Using customized browsers has disadvantages as stated in the Introduction section. The ComMentor demo we
observed depends on the remote display feature of X11 (known as
xhost). This has two disadvantages: first of all, PC users without
X-server simply cannot display the browser. Second, letting remote
access of users's display leads to many possible security holes: one
can use commands like xwd to dump a user's screen so that important
information may be leaked; one can also write a X program to steal all
the keystrokes; etc.
Through this paper, we demonstrated an architecture an architecture for
integrating OODBs with WWW to support dynamically customizable views on the
web. The architecture also introduced several new ideas such as embedded
methods that are essential to support web views. We discussed various
alternatives and issues in the implementation of this architecture. We are
expecting to see more systems realizing out architecture.
Issues and Possible Implementations
Variable and Immutable data
CGI vs. Server
Updating semi-local web objects
Authentication and Security
Possible Implementations
Realization
Various users can have their own views onto the web. The system
provides support for users to organize the web pages in the way they
like to see.
Users can make annotations to the web objects they are interested
in. The comments are stored and can be viewed by other users as
well.
The system tracks all the web access originated from a view and
provides services such as adding a new web object to the objectbase
through any browser.
Pages on the web can be edited in two ways. If a web page
provides an "update" method, the method is called. Otherwise, a
modified copy is stored in the database and can be retrieved later for
further editing. 
Proxy Implementation
Related Work
O2WebGateway and many other systems (e.g., Formtek.Web) support
access from the Web to a database, but not vice versa.
They demonstrate how to interface databases on the web but don't
provide global access to web objects.
Ockerbloom@cite[OckerbloomThesisProposal] proposes an alternative
to MIME types, called Typed Object Model (TOM), that could conceivably
be employed to support web views. Objects types exported
from anywhere on the Internet can be registered in ``type oracles'',
specialized servers which may communicate among themselves to uncover
the definitions of types registered elsewhere. Web clients who happen
upon a type they do not understand can ask one of the type oracles how
to convert it into a known supertype. It could be useful to identify
web objects so that they can be modeled by OODBs more easily.
OreO is the a proxy-server based architecture providing general
purposed information filtering between web browsers and HTTP servers.
It presents a novel way to support static views and the
inter-operation of the views.
ComMentor concentrates on sharing structured in-place annotation
attached to material in arbitrary documents on the WWW. The approach
taken by ComMentor depends on specialized browsers so that more
complicated client/server communication protocol can be leveraged.
Conclusions
References
About the Authors
Jack Jingshuang Yang
Ph.D. candidate
Columbia University Department of Computer Science
500 W. 120th St. Rm. 450
New York, NY 10027
Gail E. Kaiser
Associate Professor
Columbia University Department of Computer Science
500 W. 120th St. Rm. 450
New York, NY 10027
The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the US or NYS government, ARPA, Air Force, NSF, or NYSSTF.