Developing Web Applications with Retro

An open software lightweight web application server framework built in C++, and programmed in XML and C++.

Retro is a web application server framework. That is it provides an environment in which complex web applications can be designed, built and executed. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. The software and documentation is distributed in the hope that it may be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. (GNU General Public License can be found here.) Downloads are available for Linux/Apache and for Windows/IIS.

Retro requires sets of related web applications to be described in an XML Application Set Description - ASD. Such a description specifies what applications constitute the set, and what HTML pages or other server responses constitute each application. The flow of control between these pages is also specified in the ASD, with facilities for conditional branching, looping, etc. Variables can also be defined in the ASD at the application-set, application, or page scope, so that business data can be kept out of the programming arena, and can be easily changed at a central point. The ASD allows much of the high-level structure and logic of a complex system to be defined and documented before a single page is designed, or any custom logic is programmed.

HTML pages can be generated from templates in the fashion of .asp or ,jsp pages. Alternatively, entire responses can be generated, rather like Java Servlets. The templates - .csp files - use familiar constructs to interpolate variable values and embed custom logic. However, instead of embedding script in the template, the corresponding custom logic is provided by compiled C++ code modules according to the platform, e.g. shared object libraries for a Linux based Apache server, or DLLs in the case of Windows/IIS. This separation of page design and custom logic implementation, as well as that between the higher level ASD, should generally be helpful for team efforts.

Retro provides for persistent application instances using some technique appropriate to the particular web server. It is currently implemented for the Apache web server on GNU/Linux using the Apache FastCGI module, and for Windows/IIS using ISAPI. The combination of persistent applications and the efficiency of compiled code make for high performance.

Retro embeds support for MySQL as its database system of choice, and is therefore suitable for typical 3-tier applications. However the custom logic framework is sufficiently flexible that working with other database systems should be perfectly feasible. It also provides inherent support for intelligent client-side processing, and persistent application state, based on XML data islands. It uses the Expat XML parser internally at the server side, and provides Javascript components for browser-insensitive XML handling at the client side.

Application Sets

Retro recognizes that most web facilities consist of a set of related applications. The individual applications may be discrete, or may communicate, either directly through URLs, or through a shared database. A single application set might well constitute the entire role of some web server. However, the notion of an application set will often also be convenient for the administrative separation of responsibility for development, maintenance, and operation of applications. Currently, the Retro hierarchy consists of the Application Set, which is composed of Applications, which are in turn composed of Pages. The pages don’t have to be HTML, they just represent individual responses from the web server

At the application set level, it’s possible to define ~~constants~~variables that apply to the collection of applications, and the destination of log/debug information for the application set. It’s also possible to set a variety of defaults that can be overridden as required at the application level, such as the host names for databases,~~and~~ other database parameters, and~~the paths for error and~~ ~~exit pages.~~various paths. For example:

<?xml version="1.0"?>

<MySQL host="hostname" ~~db="dbname"~~dblist="dbname1,dbname2" preload=”1”

user="username" epw="a1Bcdef3+yhg" cache="y"/>

~~<Paths exit="/AppSetName/thankyou.htm"~~

~~error="/AppSetName/errpage.htm"/>~~<Error path="/AppSetName/errpage.htm"/>

<Exit path="/AppSetName/thankyou.htm"

…...

</ApplicationSet>

Applications

An application defines a set of Pages~~, and determines where among them an application is to start, and possibly where it is to finish~~. It can override ~~database information, and exit and error page details,~~information specified at the application-set level. It can also specify variables~~and values~~ in the scope of the application. These are in a different namespace than the Application-Set variables. For example:

<?xml version="1.0"?>

<MySQL host="hostname1" dblist=”dbname” preload=”1” user="username"

epw="a1Bcdef3+yhg"/>

~~<Paths exit="/AppSetName/thankyou.htm"~~

~~error="/AppSetName/errpage.htm"/>~~<Exit path="/AppSetName/thankyou.htm"

~~<Paths exit="/AppSetName/AppName/thankyou.htm"~~<Exit path="/AppSetName/AppName/thankyou.htm"

error="/AppsetName/Appname/errpage.htm"/>

<MySQL host="hostname2" ~~db="dbname"~~db="db1,db2" preload=”1” user="another"

epw="39hjer+uj=" cache="y"/>

</Application>

…...

</ApplicationSet>

Pages

An application comprises one or more pages, each of which may require a template and/or some code to provide logic. Pages have attributes that determine whether or not they will use a Page Implementation Object (PIO), and if so, for what purposes or phases. They also determine whether the template for the page should be cached, and whether a template is to be used at ~~all.~~

all. A page may also declare itself to be the start page for its parent application

They also describe a path or paths through the application by specifying the page that is to follow ~~a particular page, subject to any required~~them, possibly subject to dynamically determined conditions. Becauseof this conditional flow control is provided at the page level, an Application definition constitutes a high-level program.

Page scope variables can also be defined.

<?xml version="1.0"?>

<MySQL host="hostname1" dblist=”dbname” preload=”1” user="username"

epw="a1Bcdef3+yhg"/>

<Exit path="/AppSetName/thankyou.htm"

<Exit ~~path="/AppSetName/AppName/thankyou.htm"/>~~

~~<Error path="/AppsetName/Appname/errpage.htm"/>~~path="/AppSetName/AppName/thankyou.htm"

error="/AppsetName/Appname/errpage.htm"/>

<MySQL host="hostname2" ~~db="dbname"~~db="db1,db2" user="another"

epw="39hjer+uj=" cache="y"/>

</Page>

</Application>

~~<XML path=”/AppSetName/xml”/>~~

…...

</ApplicationSet>

Flow of control in a Retro application

Retro finds out what its expected to do with an HTTP request by looking for supplementary path information. A typical URL submitted to the server by a Retro application page might be of the form:

http://www.retroserver.com/trivia/bin/trivia.fcgi/app1/page2?somevar=whatever

In this URL, "trivia" is a real directory (or a web-server alias for a real directory), corresponding to the application set. The real file ~~/trivia/trivia.retro~~/trivia/bin/trivia.fcgi is the executable that FastCGI starts to service its applications (or in the Win32 case, it’s \trivia\trivia.retro, and the file holds the XML that describes the application ~~set..~~set named “trivia”. The path information following ~~the~~this file name simply states the application name and the name of the submitting page. In the example there’s some query data appended to the URL as well.

This URL makes no attempt to specify what page should be processed next. It merely reports what page submitted the request. The Retro framework will determine what page is to be served up next. You make those decisions when you write the XML for the application set. In the first instance, when an application is invoked, the page name is omitted, and is inferred from the application-set description. This, the ASD, is parsed and loaded whenever an application in the set is first requested. If a persistent executable is loaded (as in FastCGI), the application-set data becomes global read-only data in that application. If there is no loading of an executable, as in the ISAPI context, read-only application-set data is stored in a look-up table in the Retro ISAPI extension. From then on, while the server stays up, the cached data is used in read-only mode.

When Retro receives such a request, the first thing it does is to parse out the variable values supplied in the GET/POST request. Retro makes no particular distinction between these two types of request, and simply places the variable name/value pairs in a symbol table for future reference.

What happens next is determined initially by the ‘phases’ attribute on the page of the supplied name, or the page designated as first. If this contains the ‘r’ flag for return processing, then the PIO for the page is loaded and cached, and its returnproc() method is called (as long as this isn’t the first call to the first page). If the PIO doesn’t have the appropriate entry point, retro reports an error.

Once any return processing has been completed, Retro determines from the page definition, using any supplied conditions, what page is to be served next. (The return processing can if necessary force an arbitrary page, but this is rather like using goto. You can’t then tell what will happen simply by knowing the values of variables and looking at the ASD.)

A sequence of operations is then possible, depending on the value of the ‘phases’ attribute on this next page:

1) Preamble (p) – general processing before the next page can be generated,

2) Set cookies (c) – set up the values for any required cookies,

3) Extra headers (h) – set up the strings for any HTTP headers other than the standard ones,

4) Prepare to make any template insertions.

5) Programmatically generate the entire response

Each of these, if specified, requires a call to the PIO for the page, so if any these flags are specified, the PIO for the new page is loaded/cached. It’s quite possible, indeed normal, for none of these to be required, in which case, if the new page doesn’t require return processing either, you don’t need to create a PIO for it.

The PIO entry points for 1, 2, and 3, as required, are then called, and the entry point for any insertions is set up. Then the standard HTTP headers are generated, followed by the headers for any stipulated cookies, and any supplementary headers you created. Following these operations, Retro is~~then~~ in a position to generate the HTML for the page from the page template, a .csp file.

The .csp file is a normal HTML file with the addition of insertion (<%ipname%>) and substitution (<%=varname%>) tags. The latter are simply replaced with the values previously populated into the variable symbol table, possibly modified and/or augmented by return and preamble processing. The former generate a call to the PIO insertions() entry point, which is free to generate HTML directly via an ostream object. The insertion method should check the supplied insertion point name and behave accordingly, returning an error message if the insertion point name is not recognized.

When the template processing is complete, the generated HTML is flushed to the client. If it is specified in the ‘phases’ for the page, the PIO postamble entry point is then called to give you an opportunity to clear up any data structures you may have created in the stages described above.

The page sent~~in this way~~ to the browser in this way will in turn submit a request embodying its qualified name (/app/page), and the sequence will be repeated as required.

Page sequencing constructs

In the example above,

page1 and page 2 are used in sequence, as indicated by the next="+" attribute on page1. The possible values for next have the following effects:

"."	Repeat the same page
“+”	Go to the next page as defined in the ASD
"-"	Go to the previous page as defined in the ASD
"^"	Go back to the first page of the application
"pageX"	Go to the named page
*varX	Go to the page named by variable varX
“cond? page2, page3”	Ternary conditional, if cond true (“1”, “y”, “Y”), page2, else page3
~~"option# page2, page3, page4"~~	~~Jump to a page depending on the integer zero based value of variable "option".~~
"option# page2, page3, page4"	Switch to a page depending on the integer zero based value of variable "option".

If there is no ‘next’ attribute, the next page will be the exit page specified for the application set, or for the particular application. It is generally assumed that this exit page does not submit a request referencing a page in the application. Nothing prohibits this, but really it is a new invocation of the application.

Template processing options

Most pages will use a .csp template, but this is not required. If you choose to do so, you can generate the entire page, or everything after the HTTP headers. Where most of the page needs to be generated, skipping the overhead associated with template interpretation may result in improved performance.

If a template is required, you may choose to parse and interpret it as required, or to parse it once, and cache a more efficiently interpretable version or further use. Cached templates will give better performance, but will of course use more memory – the usual tradeoff.

These choices are made in the page options flags. The ‘G’ option indicates that you will generate everything; ‘g’ indicates that you will generate everything after the HTTP headers. In either case a generate() entry point is required in the PIO, and this will be called once at the appropriate point to generate whatever was specified.

The ‘c’ option indicates that the template should be parsed and cached. The ‘i’ option indicates that the page is to initialize the application state mechanism as described in the next section.

Application state processing sequence

Retro supports the use of XML data islands in your HTML pages for the maintenance of application state. You can specify that you want to do this by setting the ‘state’ attribute on the Application tag in the ASD to “hxml” (XML in a hidden HTML field). The attributes of the application can also provide the name of an HTML field and Retro variable that is to be used to store the application state XML (the default name is __retro_xml).

When a page submits, the value of this field is examined. If it exists, and contains text, its value is parsed, and the results are loaded into a simple XML object model. This read/write object exists on the Retro request object, and is available through the request object to all phases of page processing.

When the page template is interpreted, the interpreter treats an occurrence of <%=__retro_xml%> (or whatever you specified as the name in the application definition) as a special case. At that point, it will generate packed XML (no extraneous whitespace, and appropriate encodings) from the XML object, and substitute the XML as the field value.

This mechanism requires your .csp file to contain a hidden variable such as:

The XML variable name appears twice, once as the name of the HTML input field (from which itXML from the page will be loaded), and again as the name of the substitution, so that the XML will be placed there during template processing. Note the single quotes around ‘<%=__retro_xml%>’. These are necessary since your XML will doubtless contain numerous double quote characters. Single quotes in the XML are encoded (").

It’s also worth noting that to prevent redundant submission of HTML form data when using XML, you should use two or more forms in your HTML document. One of these will be reserved for submission of the page, e.g.:

<input type=hidden ~~… >~~name=__retro_xml value='<%=__retro_xml%>'>

</form>

For an example, see the ~~trivia/dbops~~dbops/member application. Note that the submit button(s) will probably have to be in this same form, since browsers appear to have difficulty submitting formA from a submit button in formB.

The initial representation of your application data should be named for the application – appname.xml – ~~and should be put wherever the templates for~~you can specify its location by using the XML tag in the ASD. By default retro will look for a directory called “xml” under the root of the application ~~are located.~~ set. It will be loaded, once per application invocation, by the page that has the ‘i’ option set. If the XML variable already contains data, this step is skipped – it’s already been loaded, and the application has recycled to the same page.

Layout and structure conventions

Although you can override most of this in the ASD file, Retro has a default directory structure for ~~the~~its execution environment. For each application set there will be a top-level directory named for the application set. Under this, the directory structure should be as follows for the Apache/FastCGI and IIS versions respectively:

AppSetName

------------------------------------------------------------------------------------------------------

| | | | |

log ------------------------------- AppSetName.retro xml bin

| | | |

App1 App2 App3 AppSetName.fcgi

---------------------

| |

template PIO

AppSetName

-------------------------------------------------------------------------------------------------------

| | | | |

log ------------------------------- AppSetName.retro xml bin

| | |

App1 App2 App3

---------------------

| |

templates PIOs

In the IIS case, an ISAPI extension is loaded explicitly, using a full path/name, and retroapi.dll can be kept in the bin directory of any one of the application sets, or elsewhere.

In the ~~Apach/FastCGI case, the loaded executable,~~Apache/FastCGI case, the executable - AppSetName.fcgi - named for the application set ~~to distinguish its log records~~, will be a symbolic link to an executable - raf.exe - kept in one of the usual places, such as /usr/local/bin, and the shared library that implements most of the retro API will be kept in a similar place, e.g. /usr/local/lib. The bin directory is used to hold this executable link, where it can be given appropriate ownership/permissions.

Other than in these respects, the directory structures are equivalent. There is a subdirectory under AppSetName for each application, and under each of these directories for the application page templates, and for the PIO objects. (In each case these are loaded explicitly as required.)

The application-set directory can conveniently contain static web content file common to the set of applications. The application directories can conveniently contain static web content files particular to that application, including, but not limited to the exit HTML page and any error HTML.

ASD file reference

Tags are described here using a loose notational convention as follows:

Required attribute: attname=”attvalue”

Optional attribute [attname=”attvalue”]

Enumerated attributes “cat | dog | mongoose”

Required child tag <tagname . . . />

One or more of same name <tagname . . . />+

Optional tag (zero or one) <tagname . . . />?

Zero or more tags of same name <tagname . . . />*

Attribute values in italics are explained separately.

ApplicationSet tag

The top level tag in a Retro ASD is the application set tag. It takes the following form:

<ApplicationSet logpath=”/path/to/log/file” [name=”AppSetName”]

[verbosity=”error | warning | info | debug”]

[version=”versionstring”]

[xmlvar=”subst_here”]>

<Application . . . />+

<Error . . . />?

<Exit . . . />?

<MySQL . . . />?

<Namespaces . . . />?

<PIO . . ./>?

<Template . . ./>?

<Variable . . . />*

<XML . . ./>?

</AppSetName>

Tags of any other name are ignored.

If the name attribute is not present, it defaults to the first joint of the name of the file from which the xml file was loaded (e.g. AppSetName.retro -> name=”AppSetName”). If the verbosity attribute is not present at the application-set scope, it defaults to error. If the version attribute is not present it defaults to the empty string. If the xmlvar attribute is not present, it defaults to “__retro_xml”.

Application tag

There must be one or more Application tags, taking the form:

<Error . . . />?

<Exit . . . />?

<Page . . . />+

<PIO . . . />?

<Template . . . />?

<Variable . . . />*

</Application>

If the state attribute is not defined, it defaults to the empty string. If the xmlvar attribute is not defined it defaults to whatever is set for the application set.

Error tag

The Error tag is optional. It takes the form:

The path provided will be used to construct an error page by appending error information and “</body></html>”. It may be defined at application-set or at application scope. A definition at application scope overrides any value at application-set ~~scope.~~

scope. This is a filesystem path. A path without a leading ‘/’ or “http” will be taken as relative to the application set, e.g. path=”path/to/error.html” in application-set “trivia” will be taken as “/trivia/path/to/error.html”, or if trivia is an alias, the corresponding path.

Exit tag

The Exit tag is optional. It takes the form:

The path provided will be used as a target URL when the application submits from a page for which no “next” attribute is provided. It may be defined at application-set scope. A definition at application scope overrides any value at application-set scope. The tag is optional, a Retro application that has no exit tag, and any page with no next attribute, will attempt to load a page ~~“/index.html”.~~“/appsetname/index.html”. Note that the path here is relative to your web server root. A path without a leading ‘/’ or “http” will be taken as relative to the application set, e.g. path=”path/to/error.html” in applicatio-set “trivia” will be taken as “/trivia/ path/to/error.html”.

MySQL tag

The MySQL tag is optional in the sense that you don’t need one if your application(s) don’t use a database. It has optional attributes at application-set scope:

<MySQL [none=”anystring”]

~~[host=”hostname”]~~

               [dd=”dbNameList”] [user=”username”]host=”hostname”

               dblist=”dbNameList”

user=”username”

epw=”encoded password”

[preload=”countToPreload”]

~~[epw=”encoded password”]/>~~[cachecon=”Y”] />

The “none” attribute with any non-empty value e.g. none=”true”, overrides all other attributes here, and says this application makes no use of a database. Specifying none will marginally increase performance. Otherwise,if any of ~~these~~the non-optional attributes not defined at application-set scope must be provided at application scope. The epw attribute is a base64 encoded encryption of the specified user’s database password. The Retro utility retropwe will encode passwords, and the retro framework will decode them using the same key. You can set your own key by recompiling a small Retro module.

A dbNameList takes the form of a comma separated list:

“dbName1,dbName2,prefix*var+suffix,prefix*var+suffix”

The preload attribute is interpreted as an integer countToPreload. This number of database connections will be established when the application starts, and Retro will report an error if any of them can’t be opened at that time.  Database names of the form prefix*var+suffix are expanded by taking the literal value of “prefix”, appending the value of request variable var, then appending the literal value of suffix.  So a database specification with preload=”1” and dd=”admin,*customer+_users”, will attempt to connect to the database “admin” at start-up, and to a database acme_users (when the value of request variable customer is “acme”), at such time as the value of customer is found to be non-blank.

Namespaces tag

The Namespaces tag is optional, and takes the following form:

<Namespaces [appset=”appsetNSQ”]

                        [app=”appNSQ”]

                        [cookies=”cookiesNSQ”]

                        [page=”pageNSQ”]/>

At substitution points in pages, where you maywant to use variables defined in the ASD, you must qualify the variable name with a namespace qualifier.  The Namespaces tag allows you to choose your own qualifiers.  By default they are:

Application set scope	AS
Application scope	AP
Page scope	PG
Cookies	CK

These namespaces are accessed programmatically through the appSetVar(), appVar(), pageVar(), and cookie() methods of the Request object.  Variables derived from the GET/POST data don’t have to be qualified, and are accessed programmatically through the value() method of the Request object.

For example, your page might contain:

<h2><%=AS::company%> <%=AP::product%> (<%=PG::version%>)></h2>

which would use variables from the application-set scope, the application scope, and the GET/POSTpage variables respectively.

Page tag

Your application must define one or more Page tags, of the following form:

<Page name=”PageName” [phases=phaseSet]

                                           [options=optionSet]

                                           [next=nextSpec]

                                           [xheads=”extraHeaders”]

                                           [mimetype=”mimeType”]/>[mimetype=”mimetype”]/>

The phaseSet attribute is any combination of the character flags:

Return processing	r
Preamble processing	p
Cookie processing	c
Extra headers	h
Insertion points	i
Postamble processing	a

If no phases attribute is provided, your page will do no custom processing, only variable substitution in the template.  Youtemplate, and you do not need to provide a PIO.

The optionSet attribute is any combination of the character flags:

Cache template	c
Generate all after headers	g
Generate all	G
Initialize state at this page	i
Use this page as the start page	s

The nextSpec attribute was described under Page Sequencing Options above,above; it may take values as follows:

"."	Repeat the same page
“+”	Go to the next page as defined in the ASD
"-"	Go to the previous page as defined in the ASD
"^"	Go back to the first page of the application
"pageX"	Go to the named page
*varX	Go to the page named by variable varX
“cond? page2, page3”	Ternary conditional, if cond true (“1”, “y”, “Y”), page2, else page3
"option# page2, page3, page4"	Jump to a page depending on the integer zero based value of variable "option".

If no next attribute is provided, Retro will jump to the Exit page specified at application-set scope, or overridden at Application scope.

The extraHeader attribute allows for shorthand representation of some common headers as follows:

Content length	c
Kill the connection	k
Don’t cache this response	n

The mimetype attribute value is used in the generation of headers for the page.  It defaults to “text/html”.

PIO tag

The PIO tag is optional. It takes the form:

The path provided will be used to load page implementation objects for pages in this application. It may be defined at application-set scope. A definition at application scope overrides any value at application-set scope. If no PIO tag is present, Retro will search for templates according to the directory structure defined under Layout and Structure Conventions above. This is a filesystem path. A path without a leading ‘/’ will be taken as relative to the application set, e.g. path=”path/to/pios” in application-set “trivia” will be taken as “/trivia/path/to/pios.

Template tag

The Template tag is optional. It takes the form:

The path provided will be used to load templates for pages in this application. It may be defined at application-set scope. A definition at application scope overrides any value at application-set scope. If no Template tag is present, Retro will search for templates according to the directory structure defined under Layout and Structure Conventions ~~above.~~

above. This is a filesystem path. A path without a leading ‘/’ will be taken as relative to the application set, e.g. path=”path/to/template” in application-set “trivia” will be taken as “/trivia/path/to/template”, or if trivia is an alias, the corresponding path.

Variable tag

The Variable tag is used to define an application-set, application, or page scope variable. It takes the form:

Variables in these scopes live in their own namespaces. Variables at application-set scope are visible to all applications in the set. Variables at application scope are visible only to that application. Variables at page scope are visible only to that page. The namespace of variables defined in the ASD is distinct from the separate namespaces used for cookies and request variables.

XML tag

The XML tag is optional. It takes the form:

It is used to specify the path to XML files representing the data to be passed between pages in the applications of a particular application set. By default, Retro will look for these in .../appsetname/xml. The XML files stored in this directory should be named for the applications they relate to – e.g. myapp1.xml. Only applications whose state attribute is set to “hxml” need to have such XML files. This is a filesystem path. A path without a leading ‘/’ will be taken as relative to the application set, e.g. path=”path/to/template” in application-set “trivia” will be taken as “/trivia/path/to/template”, or if trivia is an alias, the corresponding path.

CSP Pages

CSP pages provide some of the capabilities associated with ASP or JSP pages. They don't support embedded scripts, but they do allow for generation of HTML of arbitrary complexity. The principle aim of Retro is to avoid slow interpreted code. Because they don’t allow an arbitrary mixture of script and HTML, they are usually easier to read than many ASP or JSP pages.

If your template has an entry of the familiar form:

<%=varname%>

The CSP page interpreter will look up the value of variable varname in its symbol tables and will substitute its value for the pseudo-tag. On the other hand, where an ASP page might have:

<table>

for (var i = 0; i < limit; i++) {

response.write("...");

}

</table>

a CSP page will simply have:

<table>

<%ipname%>

</table>

where ipname is the name of an insertion point. You must provide a block of C++ code in a page-implementation object - PIO - that gets used to satisfy the named insertion point. It can be argued that his rather sparse style makes for better maintainability. At least you know exactly where to look for the code that generates this part of the HTML.

Page-Implementation objects

These are objects in the sense of Unix shared objects, or in the Windows version, DLLs. You generate your PIO by adding code to a predefined shared object code fragment that implements one of the following entry points:

Preamble	const char preamble(Request , Response *);
Set Cookies	const char cookies(Request , Response , void );
Extra Headers	const char headers(Request , Response , void );
Insertions	const char insertions(Request , Response , std::ostream &html, const char ipname, void *appdata);
Generate	const char generate(Request pr, Response prs, ostream &html, void appdata);
Postamble	const char postamble(Request , Response , void );
Return Processing	Const char returnproc(Request , Response *);

These are all optional. If they are to be called, you indicate that they are needed in the ASD as previously described. They may return null, or a null terminated C style ~~error string. In the latter case Retro will either~~static error string, or the special constant PIO_ERR_INDIRECT. A non-null return value will cause Retro either to generate a simple error HTML page describing which entry point was involved and what error message it returned. Alternatively it can interpolate the error message into an error page template that you provide, provided that no response has yet been flushed to the web server. If response has been flushed, the error message is simply appended to what has been sent, with a following “</body></html>”.

You will find more on PIOs, which are the guts of Retro programming, below, in Working with PIOs.

Database support

Retro implements Connection, Result, and Row classes as a thin layer over MySQL native APIs. Retro worker threads normally cache connection objects, so they are immediately available in the context of a Retro page submission, and can be directly used to execute SQL queries/commands.

An application-set or application definition may include a MySQL tag to describe its use of one or more databases (MAX_DB_PER_APP is currently defined as 4):

<MySQL host="hostname" dblist="dbname1,prefix*var+suffix" preload=”1” user="username"

epw="a1Bcdef3+yhg" cachecon="y"/>

This specifies that the application will use two databases. The connection to the first is to be established and cached immediately (preload=”1”), and if this is not possible, then it’s a fatal error. The second connection will be established during page processing as soon as the value of var is nonblank, and then cached. It’s the application designer’s responsibility to ensure that the database name can be established before it is actually required.

The resolved form of prefix*var+suffix consists of the concatenation of an optional literal prefix, the value of the nonblank request variable var, and an optional literal suffix. Retro will check if it can construct the name as each page is processed. When it can, it will attempt to establish and cache the connection. You can see an example of this feature in the OpenTS application set. In that case, there is a separate database for the data of each client, and the name of the database can’t be determined until an administrator or a user has logged on.

If you need to use a different user/password combination in some part of the application, establish the connection in your PIO. In this case it won’t be cached. There’s an example of this in the signup application of the OpenTS application set. You can define the user name and encoded password as variables in the ASD.

Architecture

Retro consists notionally of a dispatcher that is started when the Retro application starts. The dispatcher manages a worker thread pool, allocating each request to the first available thread. From that point on, the chosen worker thread handles the request exclusively, and multi-threading synchronization issues are minimized.

To minimize inefficiencies due to contention for data objects, the worker threads are highly independent, each with its own complete set of data. The only global data items are the read-only application-set definition object, the collection of cached templates, and the log. The former exists before any worker threads start to run, and is read-only. The other two have access controlled by a mutex, so synchronization overhead is minimal.

In the Apache implementation, the server is instructed to deal with files with the .fcgi extension using the FastCGI module. The Apache FastCGI module allows for a dynamic mode, where instances of FastCGI applications are started as required when no existing instance responds in a timely fashion to a connect on its FCGI listen socket. Alternatively you can specify that it should pre-load a number of application instances. Retro is designed to work efficiently in either of these circumstances. It takes a two-tier approach to minimizing any overhead associated with loading of application instances. First it uses a relatively small executable to implement the dispatcher, with the bulk of the framework capabilities provided by shared libraries. The executable simple creates the dispatcher object, which reads the ASD for the application set, then listens for FCGI requests from the web server. The first request for an application from an application set takes the hit of loading the dispatcher and the larger shared libraries. Subsequent requirements to load instances for other application sets need only load another copy of the small executable. Specific pages need only dynamically load the (hopefully small) shared libraries that implement the PIOs. Once the latter are loaded, they are cached.

Each instance of the dispatcher can handle a band of load, corresponding to its thread pool. The FastCGI module can start more dispatchers as required. The Linux/Apache/FastCGI implementation uses POSIX threads - pthreads.

An ISAPI Extension DLL provides the IIS implementation. The web server is set up to associate files with the extension .retro (actually XML) with the ISAPI Extension. The dispatcher is set up by the DLL startup code. The ASD is read for each application set at the time of the first request, for that application-set, and the read-only definition is cached, keyed by application-set name. Basically all the overhead of loading code etc is taken as a hit by the first request for any Retro application page. As in the Apache case, PIOs are loaded on demand, and cached.

Operation

The dispatcher listens for requests, and for each request that it accepts, creating a vestigial Request object that owns an instance of the communication mechanism between it and the web server – its channel, and a reference to the application-set object. The Request object is then given to a thread that is awoken to deal with the request. It is this thread (implemented by a WorkerThread object) that is then entirely responsible for processing the request.

The thread has the Request object read from its channel to build its data dictionaries from the HTTP headers and the get/post information. It checks the application and page specification and if necessary loads the required PIO, and creates any required database connections. These may well already be in its thread specific cache.

To actually process the request, it creates a Response object. This provides an HTML output ostream object corresponding the outgoing aspect of the channel, and creates and deals with parsing of the CSP page, error reporting, etc.

Then it calls the specified entry points for the that page, feeding them with pointers/references to the Request object, Response object, HTML output ostream, etc.

The thread then has the Response object open the appropriate template file and process the template, making substitutions, and calling the insertions entry point when named insertion points are found.

Finally, it flushes its output to the server, and then calls the postamble method if one was specified. This gives you an opportunity to clean up any data structures you may have created. Then it deletes its Request and Response objects, and the worker thread goes back to sleep.

Logging, fault tolerance, errors, & debugging

Retro has quite flexible logging, error reporting, and debugging capabilities. All of these can be omitted from Retro compilations at compile time, and can also be controlled by the ASD. You can build Retro and its applications without logging when you're confident that they are stable, and this will give you maximum performance. Retro will still send the normal information to the web server log. If it's compiled with logging, Retro can be instructed in its ASD file what level of logging it is to do (error, warning, info, debug), and where it is to put its log files.

Logging is done at the level of the application set. It is capable of displaying its output at a console, or writing it to a file, using a synchronization method appropriate to the environment. In the Apache implementation, the distinction is trivial; you can simply provide a console name as a log path. In the Windows case, a named pipe is used for console output, and a Retro utility is provided to act as the receiving end of log pipes, that can display to a console, or write to a file, or both.

Retro goes to some lengths to maintain a set of viable worker threads. When it is confronted with bugs in it's own code, or in the code of PIOs, or with bad data, or some other catastrophe, it attempts to deal with these errors at two levels:

a) Before the response object has been created

b) After the response object has been created.

Before the creation of the response object, the worker thread is concerned with extracting the CGI variables and the GET/POST variables. At this stage it will respond to exceptions thrown in the C++ code, and to signals it might catch, in the most fail-safe way it can. It does this by telling its channel that it has completed processing the request, and including an error code. This stops the channel mechanism from re-trying a request doomed to failure. The thread then marks itself as dead, and exits. In the course of doing this, it will report any information it has to whatever logging mechanism has been specified. At the next opportunity, the dispatcher will start a new thread in its place.

After the Response object has been created, Retro behaves differently. At that point it will catch exceptions (signals handling remains unchanged), and will use them to compose a message. If no output has been sent to the server, this message will be interpolated into an HTML page based on the error page specified by the ASD, or simply generated by Retro. If output has been sent, and the mime type is text/html, the message will be inserted into the page it has already started to generate. It will emit corresponding log/debug messages at the same time. If the mime type is not text/html, Retro will simply tell the web server that it encountered an internal error.

Installing Retro over Apache

You'll need to have MySQL, libstdc++v3, libexpat, libz and associated headers installed on your system. The headers and libraries for these should be in a customary place (e.g. /usr/local/lib, /usr/local/include).

To run Retro over Apache, you'll also need to have installed Apache mod_fastcgi, which you can get from www.fastcgi.com.

Build Retro in the usual GNU-oriented way:

1) Un-tar retro_VERSION.tar.gz into some convenient place.

2) In the resulting top-level directory, run ./configure

3) In the top-level directory, run make

4) In the top-level directory, run make install

You might want to vary where things are installed, in which case you can call ./configure with appropriate arguments, e.g.

./configure --prefix=PREFIX

The retro executable will be placed in PREFIX/bin, and the shared library in PREFIX/lib. Default for PREFIX is /usr/local.

During the build, the shared PIO objects (shared libraries) for the Retro examples are retained in suitable directory structures under retro_VERSION. These directory structures are rooted at opents, stress, and trivia respectively. You can use Alias directives in your Apache httpd.conf to point at them to run the examples. These directory structures also contain the .csp files and application description xml files for the example applications. Symbolic links to the RAF executable are created in the raf subdirectory for each of the test applications by make install.

The required Apache configuration can be appended to your httpd.conf file. If you installed Retro in /usr/local/retro_VERSION for example, it can look as follows:

AddType application/x-httpd-fcgi .fcgi

AddHandler fastcgi-script .fcgi

Alias /dbops /usr/local/retro_VERSION/dbops

Alias /imggen /usr/local/retro_VERSION/imggen

Alias /opents /usr/local/retro_VERSION/opents

Alias /stress /usr/local/retro_VERSION/stress

Alias /trivia /usr/local/retro_VERSION/trivia

Options ExecCGI FollowSymLinks

</Directory>

Options ExecCGI FollowSymLinks

</Directory>

Options ExecCGI FollowSymLinks

</Directory>

Options ExecCGI FollowSymLinks

</Directory>

Options ExecCGI FollowSymLinks

</Directory>

Once you’ve reached that point, you can sanity check by invoking:

http://retro-host/trivia/raf/trivia.fcgi/echo?thephrase=Hello+World

Installing Retro over IIS

TBD.

Working with Page-Implementation Objects

Processing of Retro application pages notionally takes place in seven separate phases:

Preamble	Look up or calculate and set any Request variable that will be needed in the population of the page template that is about to be displayed.
Cookies	Specify what cookies should be set when HTTP headers are generated.
Extra headers	Explicitly generate any extra HTTP headers than your application might require.
Headers	Generate standard headers and append any extra headers and cookies generated above.
Process the template or generate response	Perform substitutions, and generate HTML on-the-fly at points in the page template where this is required.
Postamble	Clean up any data structures you created during the above.
Return Processing	Take whatever actions are necessary to deal with data returned by the submission of the page.

Your interaction with the Retro framework happens during these phases when Retro calls corresponding methods in your Page Implementation Object. (A shared library/DLL) The methods are as follows.

PIO Methods

The methods you may want to provide in a PIO are described in file pio.h.

const char *preamble(Request *pr, Response *prs, void **appdata);

const char *cookies(Request *pr, Response *prs, void *appdata);

const char *headers(Request *pr, Response *prs, ostream &html, void *appdata);

const char *insertions(Request *pr, Response *prs, ostream &html,

const char *ipname, void *appdata);

const char *generate(Request *pr, Response *prs, ostream &out, void *appdata);

const char *postamble(Request *pr, Response *prs, void *appdata);

const char *returnproc(Request *pr, Response *prs);

There are two objects that are directly accessible to all of the methods of PIO objects. One is the Request object, which holds the repository of variables associated with the current request, and provides methods to control the transition to the next page. The other is the Response object, on which cookie and header values can be set, and through which error information may be written.

The headers, insertions and generate methods also receive a reference to an ostream, through which the appropriate information may be written.

These methods should return null if they succeed, or a pointer to an error message on failure. The special value PIO_ERR_INDIRECT can be returned to indicate that a composite error message has been placed in the Response object.

What Happens Where

Each of the methods listed above is called by the Retro framework if it is specified in the definition of the page being processed – e.g. phases=”pchiar” in the page definition tag causes all the methods to be called at appropriate points in the processing of the page (the flags don’t need to be in order).

Preamble

Use the preamble method to do things like examination of cookie values, or the value of specific HTTP headers that may determine the future course of the application, and for recovery of data needed for the next page from the database.

You can also create some sort of application data object in the preamble, as in:

DataObject *pdo = new DataObject(. . .);

*appdata = pdo;

Retro will cache such pointers for you with a key that you supply if requested, and will appropriately synchronize object cache methods. But beyond that, you are entirely responsible for lifetime control, concurrency issues, reference counting etc of such persistent objects. The object cache is just a place to park pointers between requests.

Cookies

The cookies method is called before HTTP headers are emitted, thus ensuring that all the information required for generation of headers is available. Use the cookie methods of the Response object to manipulate cookie values.

Headers

The headers method is called before HTTP headers are emitted, thus ensuring that all the information required for generation of headers is available. All you need to do at this point is output lines of text to the supplied ostream, e.g.

const char *headers(Request *pr, Response *prs, ostream &exheads,

void *appdata)

{

exheads << “MySpecialHeader: Whatever\n”;

return 0;

}

Insertions

You must provide an insertions method in your PIO if you use the <%ipname%> construct in the corresponding template, or if the page has the ‘g’ or ‘G’ option specified.

Simply output HTML text to the supplied ostream, e.g.

const char *insertions(Request *pr, Response *prs, ostream &html,

const char *ipname, void *appdata)

{

html << “<table width=600 border=1>\n”;

for (int i = 0; i < 10; i++) {

html << “<tr>\n”;

for (int j = 0; j < 10; j++) {

html << <td> </td>

}

html << “\n</tr>\n”

}

return 0;

}

Hint: Use literal newlines in formatting the HTML (or none at all – the browser doesn’t care).

html << “<table>\n”; // Yes

html << “<table>” << endl; // No

If you use endl, you’ll force the output buffer to be flushed, and then if you subsequently encounter an error condition, the Response object won’t be able to backtrack and create a tidy error page.

Postamble

Use this method simply to clean up any application data object you created in the preamble method.

Returnproc

Presumably the page submitted data that is of some value. Use the returnproc method to do the right thing with it.

The Request Object

The request object provides access to the variable collection, a hash table in which values derived from HTTP headers, cookies, the ASD, and HTML GET/FORM fields are stored. Retro applications work largely through testing and manipulation of these values, either automatically as the GET/POST is processed, or by the developer.

Different types of variables are segregated in separate namespaces, so that you can have an HTML field name that clashes with a header or CGI variable name, and so on.

Methods for Variable Access

// Set the value of a variable - creating the variable if necessary

void set(const string &key, const string &value);

// Get a variable value

const string &value(const string &s) const;

// Get a value as a C style string

const char *str(const string &s) ;

// Get the value of a header/CGI variable – equivalent to value(“HD:”+name);

const string &header(const string &name) const;

// Get the value of a cookie – equivalent to value(“CK:”+name);

const string &cookie(const string &name) const;

// Get the value of an ASD variable

const string &asvar(const string &name) const; // value(“AS:”+name);

const string &appvar(const string &name) const; // value(“AP:”+name);

const string &pagevar(const string &name) const; // value(“PG:”+name);

Note that the result of getting the value of a variable that has never been set is a blank string.

Methods Providing Access to ASD Properties

The request object also provides access to the various values and flags specified in the Application Set Definition (ASD), or more specifically to those that relate to the current application/page.

// What application/page are we in?

const string &appName() const;

const string &pageName() const;

// Where can various files be found?

// - the CSP files

const string &templatePath() const;

// - the PIOs

const string &pioPath() const;

// What is the default exit page for the application?

const string &exitPage() const;

// Does the application use XML to maintain state, and if so, how?

bool useXML() const;

bool useHiddenXML();

bool useCookieXML();

// Where is the initial XML file

const string &xmlSrc() const;

// What variable name holds the state XML

const string &xmlVar() const;

// What phases will the current page call methods from the PIO for, and

// what page options are set

unsigned long phases() const;

unsigned long options() const;

Methods to Control the Application

The following methods deal with situations where your page processing (often the preamble or the returnproc) must override the next page specified in the ASD. Such an override can be:

To a different page,
To an alternate template for the page (same PIO)
A redirect

// Override the next page in the ASD

void setNextPage(const string &np);

// Specify an alternate template file with name other than the page name

void setNextTemplate(const string &nt);

// Force a redirect after preamble, cookies, and headers phases

void setRedirect(const string &dest);

// Check what we set the redirect to

const string &redirectTo() const;

Manipulating State XML

The following methods provide access to any state XML. You can get a pointer to the top-level SimpleXMLDomElement, and extract the XML in its current form.

// Access to the XML DOM object

SimpleXMLDOMElement *XMLTop();

// Get the current XML from the DOM

string packedXML();

Interaction with the Database

The remaining methods deal with the applications relationship with the MySQL database

// Info about the applications use of the database

bool useDB() const;

const string &dbHost() const;

const string &dbDB(int index) const;

int dbCount() const;

const string &dbUser() const;

const string &dbEpw() const;

// Get the cached database connection(s)

MySQLConnection *getDBCon() const; // gets database 0

MySQLConnection *getDBCon(int which) const;

The Response Object

The Response object provides the primary functions of transmitting the generated page or errors arising during its processing back to the web server, and hence to the browser.

It also provides the capability to set cookies and extra headers.

Output Methods

// The primary output

ostream &html();

// The error stream - to the web-server log

ostream &errors();

// An ostream to marshal error messages

ostream &errStream();

Manipulating cookies

// Set a cookie, optionally using explicit time string like

// Sat, 22-Apr-2000 00:00:00 GMT

void setCookie(const string &name, const string &value,

const string &expire = emptyString,

const string &path = emptyString,

const string &domain = emptyString,

bool bsecure = false);

// Set a cookie using a time difference like TimeDiff(30, TD_DAYS)

void setCookie(const string &name, const string &value, TimeDiff td,

const string &path = emptyString,

const string &domain = emptyString,

bool bsecure = false);

void setSessionCookie(const string &name, const string &value);

// Delete a cookie at the browser (set it to a time in the past)

void deleteCookie(const string &name);

Response Object State

// This is probably not required in any of the PIO phases, but it tells where

// the response object thinks it is in terms of processing stages (see

// enum CSP_state in response.h

int getState() const;

Error Reporting in PIOs

Errors are reported by PIO methods in one of two ways. If your error message is a simple static string, simply return a pointer to the error string from the method.

Alternatively, if your error message must be constructed from transient data, send it to the ostream provided by the Response object errStream() method, and return the value PIO_ERR_INDIRECT which is defined in response.h.

So, if it’s just a plain old constant string:

If (stupidinput)

return “That input was stupid – pull yourself together”;

But if you want to construct something more complex:

If (requested > allowed) {

prs->errStream() << “You asked for “ << requested << “, but we only have “ << allowed;

return PIO_ERR_INDIRECT;

}

There is no significant penalty for using the second form consistently. Either way, Retro will wrap the error message with text stating which PIO method originated the error.

A non-zero return from any of the phase methods will terminate processing of the request. If no output has been flushed to the web server (all output so far has been buffered), the output is simply thrown away, and an error page is emitted. If your ASD defines an error page for the application, this is used to construct the error page. It should be of the form:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

<html>

<head>

<style>

</style>

</head>

<body>

<%errmsg%>

<!—The rest of your content here -->

</body>

</html>

Lines should be restricted to 1024 characters. Longer lines won’t crash anything, but may cause the error message to be omitted. The line with text “<%errmsg%>” should contain just that, though leading whitespace and trailing garbage will be ignored. Retro’s error message is substituted for that line.

Otherwise the emitted error page is:

error_message

</body></html>

If output has been flushed to the web server, and the content type is text/html, then:

error_message

</body></html>

is simply sent after whatever has already been flushed. The effect may be somewhat unpredictable, but any problem information is usually better than none.

Building PIOs

There's a PIO outline file in the Retro top-level directory, which is reproduced here:

#include <request.h>

#include <response.h>

#include <retrostring.h>

// During debugging!!

#define LOG 1

#include <util.h>

#include <pio.h>

// If you're using the database

#include <connection.h>

#include <result.h>

#include <row.h>

/*****************************************************************************

Use this entry point to prepare for display of the next page

Typically:

1) Perform any database operations necessary to display the next page

2) Set the values of variables required for substitutions when the template

is processed

2) Set up any data structures you'll need to perform insertions when the

template is processed

******************************************************************************/

const char *preamble(Request *pr, Response *prs, void **appdata)

{

*appdata = new MyDataObject(...);

return 0;

}

/*****************************************************************************

Use the methods of the Response object to set any required cookie values.

******************************************************************************/

const char *cookies(Request *pr, Response *prs, void *appdata)

{

return 0;

}

/****************************************************************************

Explicitly generate any further HTML headers that your application requires.

Retro will add the final extra newline.

******************************************************************************/

const char *headers(Request *pr, Response *prs, ostream &html, void *appdata)

{

html << "MyHeader: Whatever\n";

return 0;

}

/*****************************************************************************

Use this entry point to provide for points in your template where you have

<%insertion-point-name%>

The name of the insertion point is passed as an argument to the entry point call.

******************************************************************************/

const char *insertions(Request *pr, Response *prs, ostream &html, const char *ipname, void *appdata)

{

if (!strcmp(ipname, "my-ip1")) {

html << "Stuff for insertion point my-ip1";

} else if (!strcmp(ipname, "my-1p2")) {

// and so forth

} else

return "Unrecognized insertion point name";

return 0;

}

/*****************************************************************************

Clean up after yourself.

******************************************************************************/

const char *postamble(Request *pr, Response *prs, void *appdata)

{

if (appdata)

delete (MyDatObject *) appdata;

return 0;

}

/*****************************************************************************

Use this entry point to process data submitted by the page. (When you use a URI

like http://www.retroserver.com/appset/raf/apset.fcgi/app1, this does not get called.

Typically:

1) Recover the relevant parts of the data and do database updates.

2) If something went wrong, force a return to the same page

(pr->setNextPage("whatever"))

******************************************************************************/

const char *returnproc(Request *pr, Response *pi)

{

return 0;

}

You can use the Makefile.in from the trivia_pio example directory as a template for your Makefile.in. All you should need to change are the couple of lines at the top:

# These will vary from application to application

APPROOT = myapp

PIOS = page1.so page2.so

Hello Retro - Echo

This is not quite ‘hello world’, but it can be if you like. It’s going to be invoked by a URL like:

http://retro/trivia/bin/trivia.fcgi/echo?thephrase=Hello+World

Here “retro” is a placeholder for the machine that’s hosting the web server. We’re going to be working in /home/me/retro/trivia, and we’ll alias the path /trivia to this actual path in Apache’s httpd.conf. So we’ll just refer to it as /trivia.

It’s a Retro application so it must have an ASD (Yes, it’s something of an overhead for a big complex application like this, but there you go.) The ASD for our set of trivial applications lives in /trivia/trivia.retro, and for the moment is:

<?xml version="1.0"?>

</Application>

</ApplicationSet>

The required directory and file structure (files in italics) is:

/trivia

trivia.retro

/trivia/bin

trivia.fcgi (a symbolic link to /usr/local/bin/raf.fcgi)

/trivia/echo

/trivia/echo/template

doit.csp

No PIO phases are specified, so we won’t need a PIO or a directory to contain it. All we need is a template – doit.csp (templates are named for pages):

<html>

<body>

</body>

</html>

In /trivia/bin, create a symbolic link called trivia.fcgi to /usr/local/bin/raf.fcgi – making sure that it is executable by Apache. At this stage, also make sure you’ve got libretro.so in /usr/local/lib, or wherever it should go on your system. Make install should have taken care of that. You also need to have configured Apache to understand FastCGI.

Now open another shell session, and run the tty command to find out which one it is. Our ASD assumes it’s /dev/pts/1. Change the ASD if it’s something else. Assuming it’s /dev/pts/1, make it writeable using:

chmod 0666 /dev/pts/1

Invoke the application using the URL we set out at the beginning of this section. You’ll notice that the first time, the performance is awful. To display your small piece of text, you had to load a quite large shared object, wait for Retro to parse the ASD and create its worker threads, and parse and cache the page template (page definition said options=”c”). After that, however, if you send the same request again the response should be pretty quick. You should also see some log output to the new shell window that you opened.

Hello Retro - EchoUpr

The next application requires less effort, since we’ve done most of the groundwork. It will be part of the trivia application-set, so we’ll use the same ASD, and just add an application:

</Application>

It can use the same template file, but this time we’ve specified a preamble, so we will need a PIO. Our directory/file structure will have to grow to:

/trivia

trivia.retro

/trivia/bin

trivia.fcgi

/trivia/echo

/trivia/echo/template

doit.csp

/trivia/echoupr

/trivia/echoupr/template

doit.csp

/trivia/echoupr/pio

doit.so

The PIO in this case is pretty minimal. We’ll just modify the single value we got from the query string. The rest of the stuff from the outline PIO has been omitted for clarity.

#include <request.h>

#include <response.h>

#include <retrostring.h>

#define LOG 1

#include <util.h>

const char *preamble(Request *pr, Response *prs, void **appdata)

{

string s(pr->value(“thephrase”));

dd(DBG, “We executed the preamble, with value: ” << s)

pr->set(“thephrase”, toUpper(s));

return 0;

}

This contains a log macro at DGB verbosity just to illustrate how to use them. The second argument to the macro can be an arbitrary sequence of values to send to that log entry though an ostream.

All we have to do here is to make sure that our input variable is converted to upper case before the template gets interpreted – impressive, uh? Note that retrostring.h defines a number of useful methods that take a const string& (std::basic_string<char>) argument, such as toUpper().

Build the shared object, then invoke

http://retro/trivia/bin/trivia.fcgi/echoupr?thephrase=Hello+world

You should see the upper-case result in the browser, and the debug message in console /dev/pts/1.

An Error-Handling Illustration

This example is also a part of the trivia application set:

</Application>

We don’t need a template for this example, so the directory structure becomes:

/trivia

trivia.retro

/trivia/bin

trivia.fcgi

/trivia/echo

/trivia/echo/template

doit.csp

/trivia/echoupr

/trivia/echoupr/template

doit.csp

/trivia/echoupr/pio

doit.so

/trivia/error

/trivia/error/template

/trivia/error/pio

doit.so

We need two PIOs, one for the “fail” page, and one for the “crash” page. They both provide just the preamble method – fail has:

const char *preamble(Request *pr, Response *prs, void **appdata)

{

return "Something went badly wrong!";

}

crash has:

const char *preamble(Request *pr, Response *prs, void **appdata)

{

string s(pr->value("action"));

dd(DBG, s)

if (s == "throw")

throw "Something went badly wrong!";

else {

// provoke a memory access violation

char *p = (char *) 1;

*p = 0;

}

return 0;

}

The business of selecting what happens is actually handled by the error page. The significant bit of this is:

<h2>This is the error page for the /trivia/error example</h2>

<%errmsg%>

<p> 

<input onclick="document.f0.action.value='throw'" type=submit

value=" Throw ">

<input onclick="document.f0.action.value='crash'" type=submit

value=" Crash ">

</form>

This contains the usual interpolation point for the error message, but it goes on to offer two submit buttons that allow us to continue to other types of error. The “fail” page preamble simply reports an error in the way it should. This causes the error page to be displayed with the wrapped message:

Retro error:
The preamble method of the error.fail application object returned the

error message:
Something went badly wrong!

If you click the “Throw” button displayed by this page, then the preamble of the “crash” page will throw the same message. Retro will catch this, and again invoke the error page. This time though it can’t wrap it with context information, and you’ll just see:

Retro error:
Something went badly wrong!

Other types of exception that Retro might catch will be reported in the same way with as much information as can be gleaned – experiment with the preamble method to investigate these behaviors.

If you click the “Crash” button, the preamble method will provoke a memory access violation. Retro will catch the signal, and will attempt to emit a log record noting it. The browser will receive a 500 Internal Error response from the web server. It isn’t reasonable after we’ve caught a signal to assume we can press on and send error HTML.

Using Insertions

So far we’ve just used simple substitutions into a template file. However in many cases it will be necessary to generate content that is more flexible and dynamic. In these cases, you’ll probably want to generate HTML on the fly, and use insertions. The next application will generate an arbitrary table of m rows and n columns.

We’ll ad this as part of our trivia application set, so once again we have a good start. Add the new application:

</Application>

Expand the directory/file structure to:

/trivia

trivia.retro

/trivia/bin

trivia.fcgi

/trivia/echo

/trivia/echo/template

doit.csp

/trivia/echoupr

/trivia/echoupr/template

doit.csp

/trivia/echoupr/pio

doit.so

/trivia/mntab

/trivia/mntab/template

render.csp

/trivia/mntab/pio

render.so

creating the template /trivia/template/render.csp:

<html>

<body>

<h3>Enter a query string like: http://retro/trivia/bin/trivia/fcgi/mntab?rows=20&cols=10</h3>

<%trows%>

</table>

</center>

</body>

</html>

This calls for an insertion, the insertion point name being “trows”, so the PIO in this case has a little more to do:

#include <stlib.h>

#include <request.h>

#include <response.h>

#include <retrostring.h>

#include <pio.h>

#define LOG 1

#include <util.h>

const char *insertions(Request *pr, Response *prs, ostream &html,

const char *ipname, void *appdata)

{

if (strcmp(ipname, "trows")) {

prs->errStream() << “Unknown insertion point: “ << ipname;

return PIO_ERR_INDIRECT;

}

int m = atoi(pr->str(“rows”));

int n = atoi(pr->str(“cols”));

int k = 0;

for (int i = 0; i < rows; I++) {

html << “ <tr>” << endl;

for (int j = 0; j < cols; j++) {

html << “ <td> ” << k++ << “ </td>” << endl;

}

html << “ </tr>” << endl;

}

return 0;

}

The URL to invoke the mntab application is of the form:

http://retro/trivia/raf/trivia.fcgi/mntab?rows=20cols=10

Adding Cookies

Let’s say we’ve decided that if a user invokes the mnrows application with no query string, she should get the same matrix as the last time she used it, or a default 16x16 matrix if it has never been used. The PIO code becomes:

#include <stlib.h>

#include <sstream>

#include <request.h>

#include <response.h>

#include <retrostring.h>

#include <pio.h>

#define LOG 1

#include <util.h>

const char *cookies(Request *pr, Response *prs, void *appdata)

{

string rs(pr->value("rows"));

string cs(pr->value("cols"));

std::stringstream cvs;

if (rs.length() && cs.length())

cvs << rs << ',' << cs;

else

cvs << "16,16";

// create the cookie, or extend its lifetime

prs->setCookie("mnrows_cookie", cvs.str(), TimeDiff(1, TD_YEARS));

return 0;

}

const char *insertions(Request *pr, ostream &html,

const char *ipname, void *appdata)

{

if (strcmp(ipname, "trows")) {

prs->errStream() << “Unknown insertion point: “ << ipname;

return PIO_ERR_INDIRECT;

}

int m = 16;

int n = 16;

string rs(pr->value("rows"));

string cs(pr->value("cols"));

string ss(pr->cookie("mnrows_cookie"));

if (rs.empty() || cs.empty()) {

if (ss.length()) {

std::stringstream cvs(ss);

cvs >> m;

cvs.get();

cvs >> n;

}

} else {

m = atoi(rs.c_str());

n = atoi(cs.c_str());

}

int k = 0;

for (int i = 0; i < rows; i++) {

html << “ <tr>” << endl;

for (int j = 0; j < cols; j++) {

html << “ <td> ” << k++ << “ </td>” << endl;

}

html << “ </tr>” << endl;

}

return 0;

}

The cookie get and set functions are in request.h and response.h respectively.

Stressing Retro

The next test application was created so I could generate load, and generally stress Retro. It does this by the simple subterfuge of having a body.onLoad handler that immediately submits the page as soon as the browser has loaded it. This can generate some pretty savage thrashing.

The directory/file structure is:

/stress

stress.retro

/stress/bin

stress.fcgi

/stress/minimal

/stress/minimal/pio

count.so

/stress/minimal/template

count.csp

For this purpose, we’ll create a separate application-set (stress.retro), defined initially as follows:

<?xml version="1.0"?>

</Application>

</ApplicationSet>

Note that we have the verbosity set to error. Otherwise we could flood the log file/console with a load of output which would partly negate the purpose of the test. We’ll actually create this application so that we can test with an interpreted template, a cached template, and with a completely generated response. The template is as follows:

<html>

<head>

function OnLoad()

{

document.f0.submit();

}

</script>

</head>

</form>

</body>

</html>

The PIO is dual purpose, to deal with the cases when we use a template, and when we generate the entire response:

#include <stdio.h>

#include <time.h>

#include <request.h>

#include <response.h>

#include <pio.h>

#define LOG 1

#include <util.h>

// This is for the template version

const char *preamble(Request *pr, Response *prs, void **appdata)

{

char buf[20];

int ni = atoi(pr->value("iterations").c_str());

sprintf(buf, "%d", ni+1);

pr->set("iterations", buf);

return 0;

}

// This is to generate the complete page

const char *generate(Request *pr, Response *prs, ostream &out, void *appdata)

{

int ni = atoi(pr->value("iterations").c_str()) + 1;

out << "Content-type: text/html\n\n";

out << "<html><head><script>function OnLoad() { document.f0.submit(); }“;

out << “</script></head><body onLoad=\"OnLoad()\">";

out << "<form name=f0 action=\"/stress/bin/stress.fcgi/minimal/it\" ”;

out << “method=POST>";

out << "<input type=hidden name=iterations value=" << ni <<

"</form><h2>" << ni << "</h2></body></html>";

return 0;

}

If you run this application in three or four instances of the browser on a couple of machines, you can give Retro, and the web server that spawns it a pretty good beating. To complete the investigation, change the ASD as follows to generate the output:

<?xml version="1.0"?>

</Application>

</ApplicationSet>

We’ve removed the phases attribute from the count page, and changed the options to “G” – generate everything. It would be possible to go a little further by modifying the PIO to double buffer the generated HTML, and thus determine its size, and then to emit a Content-length header before content type, followed by the buffered HTML.

Database Operations and Application State

The next example uses a MySQL database, and also illustrates how you can maintain application state using XML data islands. Once again, it’s a single-application, single-page application-set, defined as follows:

<?xml version="1.0"?>

<MySQL host="localhost" dblist="retro" user="retro" preload="1"

epw="ZJMeQhB3AcQ="/>

</Page>

</Application>

</ApplicationSet>

You will need to alter the MySQL tag, at the least to set the encoded password of you choice.

The template creates a simple HTML form, with multiple submit buttons that, when clicked, set a hidden field to note what kind of action is required. Other than that, they submit to the same URL, embodying the application and page name as usual. The new elements in this example concern handling of XML data that maintains application state. The relevant parts are:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">

<html>

</script>

var xmldata;

function OnLoad()

{

xmldata = new xml_om(document.f1.__retro_xml.value);

var msg = xmldata.getValue("error.msg");

xmldata.toform(document.f0);

if (msg.length)

alert(msg);

}

function OnSubmit()

{

//check values here

//if (nogood)

// return false;

xmldata.fromform(document.f0);

document.f1.__retro_xml.value = xmldata.xml();

return true;

}

</script>

</head>

<h1><%=AS::organization%> <%=AP::title%> Example - <%=PG::element%></h1>

</form>

 <input onClick='document.f1.action.value="u"' type=submit

value=" Update ">

 <input onClick='document.f1.action.value="i"' type=submit

value=" Insert ">

 <input onClick='document.f1.action.value="x"' type=submit

value=" Quit ">

</form>

<i>Page generated at: <%=timenow%></i>

</body>

</html>

The first thing to note is that there are two script elements. The first downloads some Javascript from the server. This file, a Retro component – simplexmlom.js – implements a Javascript object that provides a simple XML parser and object model. This can be used with any browser that supports reasonably current Javascript to ensure portability of the XML data islands approach. The second script block uses methods of the object to create the XML object model from XML in the hidden variable __retro_so in form f1, populate the fields of form f0 with data, and later, to recover the data from form f0, and place it back in the hidden field in form f1. Another minor feature is that the OnLoad() function checks the value of one of the XML attributes, and if it is non-blank, will display an error message as soon as the page has been loaded.

Note the dual occurrence of “__retro_xml” in the line defining the hidden field:

This is essential to the working of Retro’s application-state mechanism. The other hidden field records which submit button was pressed, so that we know what is to be done for any particular submission in the PIO.

The XML used by this application is as follows:

<?xml version="1.0"?>

<member_data>

<address last="" first="" mi="" address1="" address2="" city=""

state="" zip=""/>

</member_data>

In the PIO, the operations of interest are as follows. First we must pick up the database connection that Retro has created for us as specified in the ASD. Then we get a pointer to the SimpleXMLDOMDocument object also created for us by Retro from the contents of the __retro_xml variable. We use the information from the DOM to build an SQL query:

MySQLConnection *pcon = pr->getDBCon();

SimpleXMLDOMDocument *pxml = pr->XMLDoc();

SimpleXMLDOMElement *pm = pxml->getChild("address");

if (!pm)

return "Bad XML data - no address";

SimpleXMLDOMElement *pinfo = pxml->getChild("info");

if (!pinfo)

return "Bad XML data - no info";

pxml->setValue("error.msg", "");

std::stringstream ss;

if (op == 'u') {

ss << "update members set first=\'"

<< sqlEncode(pm->attrib("first")) << "\', mi=\'"

<< sqlEncode(pm->attrib("mi")) << "\', address1=\'"

The sqlEncode() method performs the customary doubling of any single quotes in the attribute values, so that they are properly interpteted in the generated SQL. Then we perform the query, and poke any error message resulting from this process into the XML. Note that the MySQL API wrapper object methods throw a const char * exception if they encounter a problem.

try {

Result res = pcon->Query(ss);

if (pcon->GetAffected() != 1) {

// Report the outcome in the XML

pxml->setValue("error.msg",

"Failed to update member record in database");

}

catch (const char *err) {

pxml->setValue("error.msg", err);

}

The Simple XML Object Model – Client Side

The Javascript object implemented by simplexmlom.js provides the following methods:

Method	Description
xml_om(src)	Constructor - creates an xml_om object from a string containing XML, as in var om = new xml_om();
om.xml()	Returns a string comprising the XML currently represented by the object.
om.getValue(path)	Returns a string, which is the value of the attribute or tag represented by the string path, or null on failure. Path is a jointed string of the form: “Tag.tagchild1.tagchild2.attribute” Or: “Tag.tagchild1.tagchild2.” In the former case, getValue will return the value of an attribute, in the latter case, the text within the most deeply nested tag, less leading and trailing whitespace. See notes about arrays below.
om.getAttributes(path)	Returns an array of strings comprising the attribute values of the element represented by the path, as in “tagtop.tagchild1.tagchild2”. The strings are in the same order as the attributes were defined in the XML. Returns null on failure.
om.setValue(path, v)	Set the value of the attribute or tag represented by path (as in getValue()) to the string corresponding to v. Returns true on success, false otherwise.
om.setAttributes(path, a)	Sets the existing attributes of the element represented by path to the first n values of array a, converted to strings, where n is the number of attributes on the element. Returns true on success, false otherwise.
om.cloneArrayElement(path)	Creates a new element identical to the one specified by path. The order of insertion is not specified or implied, since the resulting elements are identical. Returns true on success, false otherwise.
om.deleteArrayElement(path)	Deletes the element (part of an array) designated by path. Returns true on success, false otherwise.
om.listLength(path)	Returns the number of elements that are the immediate children of the element represented by path, or –1 if the path is not found.
om.addElement(path, name, after)	Adds a new element to the model within the element represented by path, and following the child named by after. If after is null or blank, the new element is inserted before any existing elements. If path is blank or “.” This method inserts an element into the top-level element. Returns true on success, false otherwise.
om.addAttribute(path, name, value)	Adds an attribute to the element corresponding to path, at the end of the current list of attributes. If the path is blank or “.” The attribute is added to the outermost element. If the attribute name is null or blank, adds element text to the specified element. Returns true on success, false otherwise.
om.xmlName()	Returns the tag name of the outermost tag.
om.ok()	Returns true if the model is in a good state, false otherwise.
om.toForm(f, ep)	Populates the fields of the HTML form represented by f. The field names should represent paths as described for setValue(). The optional argument ep specifies an exclusion prefix. Fields with names prefixed by ep are ignored in this process. The default exclusion prefix is “__” (double underscore).
om.fromForm(f, ep)	Takes values from the HTML form represented by f, and populates them into the model. The field names should represent paths as described for setValue().The optional argument ep specifies an exclusion prefix. Fields with names prefixed by ep are ignored in this process. The default exclusion prefix is “__” (double underscore).

The XML Object model is simple in the sense that it knows nothing about CDATA, namespaces, entity references, processing instructions, and so on. It understands elements and attributes, and simple text as the body of an element. It does however recognize the special case of elements that contain a number of children which all have the same tag name, i.e. elements that represent arrays. However, there must be no elements of different name within the same containing element. To understand this, let’s use an example:

<?xml version="1.0"?>

The grass is greener

</child1>

</child2>

</child4>

</outer>

Here child1 is an element with some body text, and some attributes. To get the value of attr1, we would use a path “child1.attr1”, to get the value of attr2 we would use path “child1.attr2”. To get the value of the body text (less the extraneous whitespace that come before and after), we would use the path “child1.” - essentially signifying the nameless attribute. The jointed paths consist of a number of tag names followed by one attribute name. Methods that relate only to elements take a path that consists only of tag names. The outermost tag name (“outer” in this case) is never used in paths.

Child2 is a two-element array holder. It can still have attributes, and these are accessed in the same way as those of child1. To refer to attributes of the array elements, we would use path “child2.elem[0].attr1” for example. The child elements can have structures nested under them, and the model doesn’t care if they have identical structures, so you can have arrays of dissimilar objects, as long as their outer name is the same, and as long as you’re certain you know what you’re doing (it’s not recommended). To preserve simplicity in the Javascript object, arrays must have an array holder. (This can be the XML outer element, but then your data structure at the top level can only be an array, with no elements of other names.)

Child3 is similar to child1, except that it doesn’t have any body text. Child4 has contained elements (with paths “child4.inner1”, and child4.inner2”). The attributes of inner1 are reached via paths such as “child4.inner1.attr1”.

If the name of the outer tag is of interest, the object’s xmlName() method gives access to it.

There is a test script – jsomtest.js – in the retro2 top-level directory that illustrates the use of the methods except for toForm() and fromForm(). These are illustrated on the dbops.members example.

It may be useful to know that the XML generated by Retro encodes attribute values so that for instance, the following transformations occur:

O’Flannagan => O'Flannagan

This is necessary because the XML can contain no instances of the single quote character. Remember we include the XML in the hidden field using single quotes:

Value=’<%=__retro_xml%>’

The Javascript object decodes attributes automatically. However there is no corresponding encoding when XML is generated from the object model. This isn’t necessary because we don’t have to express that value in the HTML.

The Simple XML Object Model – Server Side

The SimpleXMLDOMElement class provided by the Retro API provides the following methods (the header files in retro_so are of course definitive):

Method	Description
void setText(const string &s);	Sets the body text of the element.
void streamout(ostream &, int indent = 0) const;	Formats the XML represented by the element into an ostream.
void packXML(ostream &);	Formats the XML represented by the element into packed XML (no extraneous whitespace).
const string &tagName() const;	Returns the tag name of the element.
int attribCount() const;	Returns the number of attributes on the element.
const anvpair *attributes() const;	Returns an array of attribute name value pairs – Struct anvpair { string m_name; string m_value; };
const string &attrib( const string &atname, bool *pb = 0) const;	Returns the value of attribute atname. Returns the empty string if atname doesn’t exist. If pb points to a bool value, this will receive false if the attribute doesn’t exist.
int childCount() const	Returns the number of elements that are immediate children of this element.
SimpleXMLDOMElement ** children() const;	Returns an array of pointers to the children of this element, or null if there are no children.
SimpleXMLDOMElement * getChild(const string &path) const;	Returns a pointer to a single child element by jointed name.

The SimpleXMLDOMDocument class is derived from SimpleXMLDOMElement, and provides the following methods:

Method	Description
bool good() const;	Returns true if the document object is valid.
Bool insertElement( const string &path, const string &name, const string &after, const char **attributes);	Inserts a child element into the element specified by the jointed name path (“” or “.” For the top level). The inserted element will have tag name “name”, and will be inserted after the element by “after”. If after is blank, the new element will become the first element. The list off attributes consists of alternating name and value strings, terminated by a null.
bool deleteElement( const string &path);	Deletes the element specified by the jointed name path.
bool duplicateElement( const string &path);	Duplicates the element specified by the jointed name path.
bool addAttributes( const string &path, const char ** attribs);	Adds attributes to the element specified by the jointed name path. The list off attributes consists of alternating name and value strings, terminated by a null.
bool setValue( const string &path, const string &value);	Sets the value of an attribute described by the jointed name path to value. If there is an empty last joint, as in “a.b.c.” the element text is set.
bool setValue( const char path, const char value);	Sets the value of an attribute described by the jointed name path to value. If there is an empty last joint, as in “a.b.c.” the element text is set.
const string &getValue( const string &path, bool *pbfound = 0);	Gets the value of an attribute described by the jointed name path. If there is an empty last joint, as in “a.b.c.” the element text is retrieved. If the attribute doesn’t exist, an empty string is returned. You can find out by passing a valid bool pointer.
const string &getValue( const char path, bool pbfound);	Gets the value of an attribute described by the jointed name path. If there is an empty last joint, as in “a.b.c.” the element text is retrieved. If the attribute doesn’t exist, an empty string is returned. You can find out by passing a valid bool pointer.
string packedXML();	Returns a string comprising the XML that represents the current state of the document, with encoding suitable for use in HTML.
bool readXML(const char *file);	Reads an XML file and populates the document.
bool loadXML(const char *src);	Loads XML from a string and populates the document

The methods provided by these classes are close to those provided by the Javascript model. Where there are differences these are generally aimed at better performance, e.g. setting of multiple attributes when an element is inserted.

The use of some of these methods is illustrated by the trivia/makexml example. This example also shows shows how to use Retro to produce web content that is of mime type other than text/html.

The example adds another application, and an XML path to the trivia ASD:

<?xml version="1.0"?>

...

<Page name="makexml" options="sig" next="" xheads="c"

mimetype="text/xml"/>

</Application>

</ApplicationSet>

The single page of this application specifies that it is the start page, it is to initialize “hxml” state (though we won’t actually use this), and that it will generate everything except the headers. It also stipulates that a content-length header is to be emitted, and that the type of the generated response is to be “text/xml”. I’m assuming you’ll look at the response using a browser that understands XML. Because we are to generate the output (the XML we used as an example for the Javascript XML model), we won’t need a template. However, a PIO implementing the generate() entry point is required. Here’s its definition:

const char *generate(Request *pr, Response *prs, ostream &out, void *appdata)

{

const char *al[] = { "attr1", "value1", "attr2", "value2", 0 };

SimpleXMLDOMDocument *pxml = pr->XMLDoc();

pxml->addAttributes("", al);

pxml->insertElement("", "child3", "", al);

pxml->insertElement("", "child2", "", al);

pxml->insertElement("", "child1", "", al);

pxml->insertElement("", "child4", "child3", al);

pxml->insertElement("child2", "elem", "", al);

pxml->insertElement("child4", "inner2", "", al);

pxml->insertElement("child4", "inner1", "", al);

SimpleXMLDOMElement *pe = pxml->getChild("child3");

pe->setText("The grass is greener");

pxml->setValue(“child4.inner2.attr2”, “something else”);

out << pxml->packedXML() << '\n';

return 0;

}

The XML provided for initialization of state is:

<?xml version="1.0"?>

The example cheats for the sake of simplicity by having identical attribute names and values for all the element. It will be left as an exercise for the reader to produce a more interesting example.

First we add attributes to the “outer” tag, which we can address by the path “” or “.”.

Then we insert three elements into the outer tag, with no ‘after’ specification, so they each become the first element in turn, hence we add them in reverse order. We then insert child4 after child3.

The remaining insertions add children to the elements we just added. Then we get a pointer to child3 by name, and set its body text, and we change the value of an attribute on one of the innermost elements.

Extra Headers

In the simplest use of Retro, the only response header generated is:

Content-type: text/html

This is not to say that this header is the only one the client will receive. The web server may choose to add headers to what retro has provided.

Retro provides two methods for adding further response headers. A page specification in the ASD can specify xheads options:

c - add a content-length header,

k - kill the connection (as opposed to keeping it alive which is the default in HTTP1.1),

n – no-cache, actually several headers.

Adding a content-length header involves more than just emitting the header. All of the response must be buffered before the header can be generated, so that the content length can be written into the header. You should allow Retro to do this one for you.

The no-cache option actually generates the following headers:

Cache-control: no-cache

Pragma: no-cache

Expires: Sat, 01 Jan 2000 00:00:00 GMT

Hopefully this cocktail will be sufficient to dissuade most browsers and proxies from using a cached copy.

You can also include extra headers explicitly if you specify the ‘h’ flag in the phases for the page, and provide the headers() entry point in your PIO. Remember to send a ‘\n’ after each header (though you can get away with the last one, Retro will check) – for example:

const char *headers(Request *pr, Response *prs, ostream &html, void *appdata)

{

html << "Date: Tue, 15 Jul 2003 16:00:00 GMT\n";

html << "Message-ID: 23ab-ff87-ca67-22ee\n";

html << "ETag: fuzzy\n";

return 0;

}

If you use this facility, you had better understand what you are doing.

Mime Type

A page specification in the ASD also allows you to specify the mime type of the response. The trivia/makexml example shows how this is used.

Because Retro lets you generate response from compiled code down a fast connection to the client, there are many mime types that could be of interest.  For example you could generate graphical material into a pixel matrix, then emit the matrix as a PNG or other bitmapped graphic file. There’s an example in application-set imggen. The ASD is as follows:

<?xml version="1.0"?>

<ApplicationSet verbosity="debug" logpath="/dev/pts/2">

   <Application name="makepng" state="hxml">

      <Page name="makepng" options="sg" next="" xheads="cn"

                                            mimetype="image/png"/>

   </Application>

</ApplicationSet>

There’s no template. The PIO uses libgd (which in turn requires libpng and libz), and is coded as follows:

#include <math.h>

#include <request.h>

#include <response.h>

#include <retrostring.h>

#define LOG 1

#include <util.h>

#include <pio.h>

#include "gd.h"

#include "gdhelpers.h"

// libgd requires a context object to do output to something other

// than a file, so we provide a thin wrapper around the out ostream

struct RetroIOCtx

  RetroIOCtx(ostream &os);

  gdIOCtx m_ctx;

  ostream &m_os;

};

// We only need the output methods

static int __putBuf(gdIOCtx *pctx, const void *src, int sz)

   RetroIOCtx *ctx = (RetroIOCtx *) pctx;

   ctx->m_os.write((const char *)src, sz);

   return sz;

static void __putChar (gdIOCtx *pctx, int c)

   RetroIOCtx *ctx = (RetroIOCtx *) pctx;

   unsigned char uc = (unsigned char) (c & 0xff);

   ctx->m_os << uc;

// Cover the free operation just in case

void __free(gdIOCtx *)

{}

RetroIOCtx::RetroIOCtx(ostream &os) : m_os(os)

   memset(&m_ctx, 0, sizeof(gdIOCtx));

   m_ctx.putC = __putChar;

   m_ctx.putBuf = __putBuf;

   m_ctx.gd_free = __free;

const char *generate(Request *pr, Response *prs, ostream &out, void *appdata)

   RetroIOCtx ctx(out);

   // First sort out the input

   int size = atoi(pr->value("size").c_str());

   if (size <= 0)

      size = 100;

   int npoints = atoi(pr->value("points").c_str());

   if (npoints <= 0)

      npoints = 10;

   int rotn = atoi(pr->value("rotn").c_str());

   rotn %= 360;

   int numerator = atoi(pr->value("num").c_str());

   if (numerator <= 0)

      numerator = 1;

   int denominator = atoi(pr->value("den").c_str());

   if (denominator <= 0)

      denominator = 2;

   int red = atoi(pr->value("red").c_str());

   if (red < 0) red = 0;

   if (red > 255) red = 255;

   int green = atoi(pr->value("green").c_str());

   if (green < 0) green = 0;

   if (green > 255) green = 255;

   int blue = atoi(pr->value("blue").c_str());

   if (blue < 0) blue = 0;

   if (blue > 255) blue = 255;

   gdImagePtr im;

   im = gdImageCreate(size, size);

   int white = gdImageColorAllocate(im, 255, 255, 255);      // background color

   int fill = gdImageColorAllocate(im, red, green, blue);

   gdImageColorTransparent(im, white);

   int i;

   double center = (double)size/2;

   double ratio = (double) numerator/denominator;

   gdPoint *tpp = (gdPoint *) alloca(npoints*2*sizeof(gdPoint));

   if (!tpp) throw "Out of memory";

   double rot = rotn;

   rot =(rot*M_PI*2)/360.0;

   double theta = rot;

   double subtend = M_PI/npoints;

   for (i = 0; i < npoints; i++) {

      tpp[2*i].x = (int) (center+center*cos(theta));

      tpp[2*i].y = (int) (center-center*sin(theta));

      theta += 2*subtend;

   theta = subtend+rot;

   double ir = ratio*center;

   for (i = 0; i < npoints; i++) {

      tpp[2*i+1].x = (int) (center+ir*cos(theta));

      tpp[2*i+1].y = (int) (center-ir*sin(theta));

      theta += 2*subtend;

   gdImageFilledPolygon(im, tpp, npoints*2, fill);

   // Output the image in PNG format

   gdImagePngCtx(im, (gdIOCtx *) &ctx);

   gdImageDestroy(im);

   return 0;

This creates a PNG image of a star using the parameters provided in the query string. The query should be of the form

?size=300&points=15&rotn=0&num=1&den=2&red=255&green=0&blue=0

Here, size is the side of the bounding square, points is the number of points on the star, rotn is the initial rotation angle, num and den specify a ratio – e.g. 1/2 - that specifies the diameter of the inner circle of the star relative to the outer circle, and red, green, and blue represent the required color. All have some default value.

The background is white and white is specified as being the transparent color.

The Retro Linux Build Environment

You might find it convenient to build your Linux Retro applications in the same environment as the examples.

To facilitate this, Retro provides a command line program to create the correct directory structure, create the required makefiles, create a link to raf.fcgi, and to set up skeletons for the ASD, an application error file, and a skeleton page PIO source and template.

To create a new application-set, in the top-level Retro directory invoke the executable as follows:

raf/pioprgen foo app1 page1

where foo is the name of your new application-set, foo1 is its first application, and foo1page1 is the first page of that application. This should set up a directory structure and files as follows:

/foo

/foo/bin

foo.fcgi (symlink)

/foo/app11

error.html

/foo/app1/pio

page1.cc

page1.d

page1.o

page1.so

Makefile

Makefile.in

pioimple.h

/foo/app1/template

foo1page1.csp

/foo/xml

foo.retro

Makefile

Makefile.in

The program does not modify your httpd.conf file. You will need to add an alias and a directory entry at this point before the vestigial application will run. Something like

Alias /foo /home/you/retro2/foo

Options ExecCGI FollowSymLinks

</Directory>

If you are using a console for debug output, check which terminal it is (tty) and make sure it’s writeable (e.g. chmod 0666 /dev/pts/1). Modify the foo.retro file to reflect the correct console or log file. You should also bounce Apache – apachectl restart.

The same program can be used with a similar command line to add an application to the application-set, or add a new page to an application.

The Windows Build Environment

You might find it convenient to build your Windows Retro applications in the same environment as the examples.

To facilitate this, Retro provides a command line program to create the correct directory structure, create the required project and workspace files, and to set up skeletons for the ASD, an application error file, and a skeleton page PIO source and template.

To create a new application-set, in the top-level Retro directory invoke the executable as follows:

Bin\pioprgen foo foo1 page1

where foo is the name of your new application-set, foo1 is its first application, and foo1page1 is the first page of that application. This should set up a directory structure and files as follows:

\foo

\foo\foo1

error.html

\foo\foo1\pio

page1.cpp

page1.obj

foo_app1_page1.dsp

foo_app1_page1.dsw

pioimple.h

\foo\foo1\template

page1.csp

\foo\xml

foo.retro

The program does not modify your IIS configuration. You will need to use the Internet Services Manager to add a virtual directory for foo, and to provide that with an association for the extension “.retro” to retroapi.dll. (See installing on Windows.) You will probably find that you also need to bounce IIS before the new application will work.

The same program can be used with a similar command line to add an application to the application-set, or add a new page to an application.

When you’re happy with what you have created, you can add it to the retro.dsw workspace if you find it convenient to have everything in the same workspace.

The OpenTS Example

OpenTS is a web application designed to allow the employees of service companies/organizations to fill in their weekly time sheet from anywhere where they can access the web. The author originally wrote an earlier CGI version of OpenTS in a fit of peek about the time sheet system used by the organization where he had his day job. It has subsequently been rewritten to investigate the ease with which 'real' applications can be built using Retro.

The outcome was some changes to Retro, (Late loading of page-implementation objects, extension of the next attribute options in application definitions, improved cookie setting provisions, some support for digest and encryption algorithms), and some degree of comfort about the ease of use of Retro. It's about the same as ASP or JSP programming, as you'd expect, though the ability to describe the application-set at high level in the ASD is a distinct advantage.

OpenTS is designed to give a reasonable level of security without the need to set up a secure server. It doesn't send plain text passwords over the wire except at initial setup of an account. Instead it uses digests of passwords (account administration), or passwords encrypted with the previous password (user password changes). For these activities it uses Javascript implementations of SHA1 and TEA (Tiny Encryption Algorithm) in client-side scripts. Corresponding facilities are provided in the Retro shared library for the server side.

For example in the account administration template script, the user must enter her password on the main menu page. A hash is then composed of a timestamp value sent by the server, and the entered password, and this is sent:

var s = document.form0.lts.value+document.form0.clientpwd.value;

document.form0.token.value = calcSHA1(s);

document.form0.clientpwd.value = "********";

The server can look up the password in the database, and recalculate the hash to validate the user. Similarly when a user changes her password, she must enter her current password, and a hash is calculated. This is then used to encrypt the new password:

var s = form.lts.value+form.pwd.value;

form.token.value = calcSHA1(s);

s = doEncipher(form.newpwd.value, form.pwd.value);

form.pwd.value = "";

The code on the sever side can then calculate the same hash, and decrypt the new password.

OpenTS makes some use of XML data objects, particularly for the actual time sheet data entry screen, but also to pass data structures to populate drop-lists of customers and their associated projects. However it doesn’t use the built-in “hxml” state mechanism – that’s how it was, and if it’s not broken, you don’t mend it.

If anyone ever uses it for real data, please let us know!