Abstract. The expressive power of Prolog enables its use as a general programming language. However, in the case of applications with web interfaces, the console-oriented, question–answer nature of the language is a considerable drawback. Even though the series of questions and answers may be embedded in a series of HTTP requests and responses, this approach, while acceptable for simpler cases, does not integrate into the web user interface model. First, the paper provides an overview of Prolog language extensions facilitating web programming available at the user's disposal, in particular, the PiLLoW library and foreign language interfaces. Based on these, it sketches a two-layered system architecture that allows constructing complex applications. On one hand, the architecture gives support to access HTTP in a transparent manner integrated into the language, to preserve state between requests, to serve concurrent requests efficiently. On the other hand, the architecture means an extensible template technology with an expression language and structured JSP-like formalism, which encourages the separation of view and business logic. It aims to make it possible to implement applications with web interfaces purely in Prolog, thereby making it unnecessary to embed the application into a system architecture with support for the web. The HTTP protocol and low-level communication is hidden from the programmer by Prolog Web Container, which connects to the web server. It provides an environment for executing user programs, marshals request handling, preserving state between requests and load balancing, thereby giving background support for the Prolog Server Pages technology. Prolog Server Pages, a simple yet extensible structural mechanism, makes it possible to unambiguously separate view and logic. The paper, utilizing its multi-threading capabilities and its built-in low-level external data access support, concludes with a reference implementation in SWI-Prolog, which realizes the presented system architecture. A sample application is provided for demonstration.

Abstract. In web search engines, collecting source documents is an indispensable function to be performed periodically at high speeds. For this end, an extensible, component-based, loosely-coupled distributed architecture for the .NET platform is presented that facilitates efficient parallel crawling. It combines flexibility key to research with scalability, which is a must for high performance. The architecture comprises of a lower layer that constitutes an execution environment and an upper layer that realizes a distributed crawler with a central coordinator.

Abstract. The Internet is one of today's primary information sources yet information available through this medium is scattered and is not necessarily present in its original storage format. In fact, information is available as a set of interconnected documents each with a possibly different terminology. Nevertheless, it is a typical task to return a set of documents that match a given query, exemplified by the wide-spread use of web search engines. Such retrieval of documents, however, is only possible through a periodic traversal of the Web.

For the traversal to be effective, that is, frequently updated documents to be visited regularly, parallelization is indispensable. However, operating a distributed system that supports parallelization across multiple machines is a substantially more complex job than operating a centralized system. Other crucial aspects include easy tracing of job completion and flexible architecture. It is essential that the system support monitoring how jobs are processed and it adopt to various crawling tasks (e.g. traversal of specific domains, language-dependent behavior, reference structure analysis).

In order to battle the aforementioned demands, the author presents an architecture comprising of units loosely-coupled by means of well-defined interfaces and communicating with one another by exchanging messages. Loose coupling allows units to run on different machines and caters for extension with new functionality. Communication between units is transparent independently of whether it is in fact a local or a remote message exchange, and the way units are interconnected is declaratively defined in XML descriptors. In general, the services offered by an underlying framework greatly reduce the complexity that would otherwise be related to coordinating a distributed system. Unit development may subsequently focus on other aspects of web robot construction.