![[graphic]](image.gif)
HTTPDMON Graphic
[next] [previous] [contents] [full-page]23.1 - Echo Facility
23.2 - Hiss Facility
23.3 - Where Facility
23.4 - Xray Facility
23.5 - Apache Bench
23.6 - CALogs
23.7 - HTAdmin
23.8 - HTTPd Monitor
23.9 - MD5digest
23.10 - QDLogStats
23.11 - SECHAN Utility
23.12 - Scrunch Utility (obsolete)
23.13 - StreamLF Utility
23.14 - WASD Bench :^)
23.15 - Server Workout (obsolete)
Foreign commands for external utilities (and the HTTPD control
functionality) will need to be assigned from the adminstration users' LOGIN.COM
either explicitly or by calling the
HT_ROOT:[EXAMPLE]WASDVERBS.COM
procedure.
$ AB == "$HT_EXE:AB"
$ HTTPD == "$HT_EXE:HTTPD"
$ HTTPDMON == "$HT_EXE:HTTPDMON"
$ MD5DIGEST == "$HT_EXE:MD5DIGEST"
$ QDLOGSTATS == "$HT_EXE:QDLOGSTATS"
$ SECHAN == "$HT_EXE:SECHAN"
$ STREAMLF == "@HT_EXE:STREAMLF"
$ WB == "$HT_EXE:WB"
23.1 - Echo Facility
Ever had to go to extraordinary lengths to find out exactly what your browser is sending to the server? The server provides a request echo facility. This merely returns the complete request as a plain-text document. This can be used for for checking the request header lines being provided by the browser, and can be valuable in the diagnosis of POSTed forms, etc.
This facility must be enabled through a mapping rule entry.
script /echo/* /echo/*
It may then be used with any request merely by inserting "/echo" at
the start of the path, as in the following example.
http://wasd.dsto.defence.gov.au/echo/ht_root/
23.2 - Hiss Facility
The hiss facility provides a response stream made up of random alpha-numeric characters (a sort of alpha-numeric white-noise). No response header is generated and the stream will continue (by default) up to one megabyte of output, or until the client closes the connection.
This facility must be enabled through a mapping rule entry.
script /hiss/* /hiss/*
Usage details are described in 7.8 - Site Attacks.
23.3 - Where Facility
Need to locate where VMS has the HTTPd files? This simple facility maps the supplied path then parses it to obtain a resulting VMS file specification. This does not demonstrate whether the path actually exists!
This facility must be enabled through a mapping rule entry.
script /where/* /where/*
It may then be used with any request merely by inserting "/where" at
the start of the path, as in the following example.
http://wasd.dsto.defence.gov.au/where/ht_root/
23.4 - Xray Facility
The Xray facility returns a request's complete response, both header and body, as a plain text document. Being able to see the internals of the response header as well as the contents of the body rendered in plain text can often be valuable when developing scripts, etc.
This facility must be enabled through a mapping rule entry.
script /Xray/* /Xray/*
It may then be used with any request merely by inserting "/xray" at
the start of the path, as in the following example.
http://wasd.dsto.defence.gov.au/xray/ht_root/
23.5 - Apache Bench
This server stress-test and benchmarking tool, as used in the Apache
Distribution, is included with the WASD package (sourced from
http://webperf.zeus.co.uk/ab.c), within
license
conditions.
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd.
Copyright (c) 1998 The Apache Group.
Apache Bench will only compile and run for Alpha, IA64 or VAX systems with VMS 7.n or greater available. Also see the WASD analogue, 23.14 - WASD Bench :^). Apache Bench is a simple but effective tool, allowing a single resource to be requested from a server a specified number of times and with a specified concurrency. This can be used to benchmark a server or servers, or be used to stress-test a server configuration's handling of variable loads of specific resquests (before exhausting process quotas, etc.)
A small addition to functionality has been made. The WASD Apache Bench displays a count of the HTTP response categories received (i.e. the number of 2nns, 4nns, etc.) This allows easier assessment of the relevance of results (i.e. measuring performance of some aspect only to find the results showed the performance of 404 message generation - and yes, an annoying experience of the author's prompted the changes!)
The following examples illustrate it's use.
$ AB -H
$ AB -C 10 -N 100 http://the.server.name/ht_root/exercise/0k.txt
$ AB -C 50 -N 500 -K http://the.server.name/ht_root/exercise/64k.txt
$ AB -C 10 -N 100 http://the.server.name/cgi-bin/cgi_symbols
23.6 - CALogs
The Consolidate Access LOGS utility (pronounced similar to the breakfast cereal brand :^) merges multiple HTTP server common and combined format access logs into a single log file with records in time-order. Due to the granularity of HTTP server entry timestamps (one second) the records are sorted to the one second but not within the one second.
It uses RMS and the VMS sort-merge routines to provide it's basic consolidation functionality. An RMS search uses the supplied wildcard log file specification. Matching files are opened and each record read. The date/time field is parsed and a binary timestamp generated. Records with formats or date/time fields that do not make sense to the utility are discarded. When all files have been processed the sort-merge is performed using the timestamp as the key. The sorted records are then written to the specified output file.
$ calogs <log-file-spec> [<output-file-name>] [<qualifiers>]
$ CALOGS == "$HT_EXE:CALOGS" $ CALOGS HT_LOGS:*200205*.LOG 2002_MAY.LOG $ CALOGS /VERBOSE HT_LOGS: $ CALOGS /NOWASD HT_LOGS:*200206*.LOG_* /OUTPUT=2002_JUNE.LOG $ CALOGS /PROXY /NOWASD HT_LOGS:*2002*.LOG 2002_PROXY.LOG
The HTAdmin utility assists in with the command-line maintenance of $HTA authorization databases (see 14.3 - Other Authentication and 15.5 - Authorization Sources).
$ htadmin <database> [<username>] [<qualifiers>]
$ HTADMIN EXAMPLE /CREATE
$ HTADMIN EXAMPLE /DELETE /CONFIRM
$ HTADMIN EXAMPLE DANIEL
$ HTADMIN EXAMPLE DANIEL /FULL
$ HTADMIN EXAMPLE DANIEL /ADD /NAME="Mark Daniel"
$ HTADMIN EXAMPLE DANIEL /ADD /WRITE /CONTACT="Postal Address"
$ HTADMIN EXAMPLE DANIEL /ADD /NAME="Mark Daniel" /PASSWORD $ HTADMIN EXAMPLE DANIEL /ADD /NAME="Mark Daniel" /PASSWORD=cher10s $ HTADMIN EXAMPLE DANIEL /ADD /NAME="Mark Daniel" /GENERATE $ HTADMIN EXAMPLE DANIEL /ADD /NAME="Mark Daniel" /PIN
$ HTADMIN EXAMPLE DANIEL /UPDATE /EMAIL="Mark.Daniel@wasd.vsm.com.au"
$ HTADMIN EXAMPLE DANIEL /UPDATE /PASSWORD $ HTADMIN EXAMPLE DANIEL /UPDATE /GENERATE $ HTADMIN EXAMPLE DANIEL /UPDATE /PIN
$ HTADMIN EXAMPLE DANIEL /UPDATE /DISABLE $ HTADMIN EXAMPLE DANIEL /UPDATE /ENABLE
$ HTADMIN EXAMPLE $ HTADMIN EXAMPLE /FULL $ HTADMIN EXAMPLE /CSV
The /SORT qualifier sorts the current database records according to the /SORT= parameters. It can be used with the /LIST qualifier to produce ordered reports or will output the records into another authentication file. By default it sorts ascending by username. Qualifier parameters allow a sort by DATE or COUNT. Each of these allows the further specification of which date or count; ACCESS, CHANGE or FAILURE.
$ HTADMIN EXAMPLE /LIST /SORT=DATE=ACCESS $ HTADMIN EXAMPLE /LIST /SORT=COUNT=FAILURE /OUTPUT=EXAMPLE.LIS
$ HTADMIN EXAMPLE /SORT
$ HTADMIN EXAMPLE /SORT /OUTPUT=ANOTHER
$ HTADMIN EXAMPLE /LIST /SORT=DATE
$ HTADMIN EXAMPLE /LIST /SORT=DATE=FAILURE
$ HTADMIN EXAMPLE /SORT=COUNT
The HTTP server may be monitored in real-time using the HTTPDMON utility.
HTTPDMON Graphic
This utility continuously displays a screen of information comprising three or four of the following sections:
The following shows example output (after WWWRKOUT server testing):
WASD:: 1 HTTPDMON v2.0.0 AXP Saturday, 12-MAY-2001 11:26:18
Process: HTTPd:80 PID: 2020022F User: HTTP$SERVER Version: 7.2
Up: 0 21:05:51.88 CPU: 0 07:34:19.99 Startup: 1
Pg.Flts: 3526 Pg.Used: 37% WsSize: 6402 WsPeak: 4729
AST: 196/200 BIO: 1003/1020 BYT: 202270/202720 DIO: 498/500
ENQ: 1989/2000 FIL: 298/300 PRC: 100/100 TQ: 26/30
Request: 184865 Current:16 Throttle: 0/0/0% Peak: 29
Accept: 178212 Reject:0 Busy:0 SSL: 17
CONNECT: 0 GET: 164717 HEAD: 20148 POST: 0 PUT: 0
Admin: 0 Cache: 48/39992/0 DECnet: 3327/6655 Dir: 6670
DCL: CLI:16730 CGI:9987 CGIplus:13310/13182 RTE:16/15 Processes: 168/5
File: 3384/0 IsMap: 0 Proxy: 64911 Put: 0 SSI: 16657 Upd: 9988
1xx: 0 2xx: 157816 3xx: 3 4xx: 18496 (403:27) 5xx: 1864 (none:0)
Rx: 19,762,345 Tx: 1,053,847,437 (bytes)
Proxy: enabled
CONNECT: 0 GET: 58720 HEAD: 6191 POST: 0 PUT: 0
Not: Cacheable Request:14555 Response:23786
Network: Rx:118,463,055 Tx:7,208,716 (29%) Accessed: 38356
Lookup: Numeric:19 DNS:21 Cache:38317 Error:1
Cache: Rx:2,863,617 Tx:320,164,012 (71%) Read:26554/3 Write:10
Device: DKA0: 4110480 blocks (2007MB) Space: available
2171744 used (1060MB 53%), 1938736 free (946MB 47%)
Purge: 18 00:00:54, 98 files (2254/2412), 0 deleted (0/0)
Time: 12 11:25:11 Status: 200 Rx: 95 Tx: 34479 Duration: 0.2400
Service: beta.dsto.defence.gov.au:80
Host: beta.dsto.defence.gov.au (131.185.250.201)
Request: GET /ht_root/exercise/64k.txt
The "/HELP" qualifier provides a brief usage summary.
The server counter values are carried over when a server (re)starts (provided the system has stayed up). To reset the counters use the online Server Administration facility (18 - Server Administration).
If [DNSlookup] is disabled for the HTTP server the HTTPDMON utility
attempts to resolve the literal address into a host name. This may be disabled
using the /NORESOLVE qualifier.
23.9 - MD5digest
From RFC1321 ...
" The [MD5] algorithm takes as input a message of arbitrary length and produces as output a 128-bit "fingerprint" or "message digest" of the input. It is conjectured that it is computationally infeasible to produce two messages having the same message digest, or to produce any message having a given prespecified target message digest. "
The MD5DIGEST utility is primarily provided with WASD for verifying kits as unchanged from the originals released. With the proliferation of mirror sites and other distribution resources it has become good practice to ensure kits remain unchanged from release, to distribution, to installation site (changes due to to data corruption or malicious intent - as remote a possibility as that may seem). Of course it may also be used for any other purpose where the MD5 hash is useful.
For verifying the contents of a WASD release connect to the
original WASD distribution site, refer to the download page,
and make a comparison between the release MD5 hash found against the list of
all archive hashes and the MD5 hash of your archive. That can be done as
follows
$ MD5DIGEST == "$HT_EXE:MD5DIGEST"
$ MD5DIGEST device:[dir]archive.ZIP
The result will look similar to
MD5 (kits:[000000]htroot710.zip;1) = 404bbdfe0f847c597b034feef2d13d2d
Of course, if you have not yet installed your first WASD distribution using
the MD5DIGEST utility that is part of it is not feasable. The original site
can provide kits and pre-built executables for this purpose.
23.10 - QDLogStats
Quick-and-Dirty LOG STATisticS is a utility to extract very elementary statistics from Web server common/combined format log files. It is intended for those moments when we think "I wonder how many times that new archive has been downloaded?", "How much data was transfered during November?", "How often is such-and-such a client using the authenticated so-and-so service?", "How much has the mail service been used?" ... and want the results in a matter of seconds (or at least a few tens of seconds ;^) It is available at the command-line and as a CGI script.
For QDLOGSTATS to be available as a CGI script it must
have authorization enabled against it (to prevent potential ad hoc browsing of
a site's logs). The following provides some indication of this configuration,
although of course it requires tailoring for any given site.
[VMS]
/cgi-bin/qdlogstats ~webadmin,131.185.250.*,r+w ;
It could then be accessed using
http://the.host.name/cgi-bin/qdlogstats
The initial access provides a form allowing the various filters and other
behaviours to be selected. The CGI form basically parallels the command-line
behaviour described below.
Filters
A number of filters allow subsets of the log contents to be selected. These filters support the same string matching expressions as the server (11 - String Matching).
A knowlege of the format and contents of the common and combined log formats will assist in deciding which and to what purpose filters should be used. Record filtering is done in the same order as is finally displayed, so method would be processed before user-agent for instance. Normally a record match terminates on the first non-matched filter (to expedite processing). To compare and report each filter for every record apply the /ALL qualifier. To view records as they are processed use the /VIEW qualifier. This by default displays all matched records, but the optional =ALL or =NOMATCH parameters will display all records, or all those but the matches.
$ QDLOGSTATS log-file-spec [pattern qualifiers] [other qualifiers]
$ QDLOGSTATS HT_LOGS:*1999*.LOG /DATE="*/SEP/1999*"
$ QDLOGSTATS HT_LOGS:*.LOG /USERAGENT=*MOZILLA*X11*
$ QDLOGSTATS HT_LOGS:*.LOG /METHOD=POST
$ QDLOGSTATS HT_LOGS:*.LOG /PATH="/cgi-bin/*"
$ QDLOGSTATS HT_LOGS:*8080*.LOG /PATH="http://*.compaq.com*"
$ QDLOGSTATS HT_LOGS:*8080*.LOG /METHOD=POST /PATH="http://*sex*.*/*" /VIEW
$ QDLOGSTATS HT_LOGS:*.LOG /AUTHUSER=DANIEL
The SECHAN utility (pronounced "session") is used by
[INSTALL]SECURE.COM and its associated procedures to make file system security
settings. It is also available for direct use by the site administrator.
See SECHAN Utility.
23.12 - Scrunch Utility (obsolete)
SCRUNCH Obsolete with 7.2
Changes with server-internal SSI document handling have made the SCRUNCH utility obsolete for WASD versions 7.2 and later. Previously SCRUNCHed documents will continue to be processed without needing to be explicitly UNSCRUNCHed.
This simple procedure used the FDL facility to convert files to STREAM_LF format. The WASD HTTPd server access STREAM_LF files in block/IO-mode, far more efficiently that the record-mode required by variable-record format files.
NOTE: The server can also be configured to automatically
convert any VARIABLE record format files it encounters to STREAM_LF.
23.14 - WASD Bench :^)
WASD Bench - an analogue to Apache Bench (23.5 - Apache Bench) Why have it? Apache Bench only compiles and runs on VMS 7.n and later. This version should compile and run for all supported WASD configurations. It also has the significant performance advantage (looks like ~25%) of using the underlying $QIO services and not the socket API, and is AST event driven rather than using the likes of select(). It is not a full implementation of AB (for instance, it currently does not do POSTs). The CLI attempts to allow the same syntax as used by AB (within the constraint that not all options are supported) so that it is relatively easy to switch between the two (perhaps for comparison purposes) if desired.
The following examples illustrate it's use.
$ WB -H
$ WB -C 10 -N 100 http://the.server.name/ht_root/exercise/0k.txt
$ WB -C 50 -N 500 -K http://the.server.name/ht_root/exercise/64k.txt
$ WB -C 10 -N 100 http://the.server.name/cgi-bin/cgi_symbols
WASD Bench also has an exercise option, functionality is not
found in Apache Bench. It is basically to supercede similar functionality
provided by the retired WWWRKOUT. The exercise functionality allows WASD Bench
to be used to stress-test a server. This behaviour includes mixing HEAD (~5%)
with GET requests, and breaking requests during both request and response
transfers (~5%). These are designed to shake up the server with indeterminate
request types and client error behaviours. The best way to utilize this
stress-testing is wrap WASD Bench with a DCL procedure providing a variety of
different requests types, quantities and concurrencies.
$!(example "wrapper" procedure)
$ IF P1 .EQS. "" THEN P1 = F$GETSYI("NODENAME")
$ WB = "$HT_EXE:WB"
$ SPAWN/NOWAIT WB +e +s +n -n 100 -c 5 http://'p1'/ht_root/exercise/0k.txt
$ SPAWN/NOWAIT WB +e +s -k -n 50 -c 5 -k http://'p1'/ht_root/exercise/64k.txt
$ SPAWN/NOWAIT WB +e +s -n 50 -c 2 http://'p1'/cgi-bin/conan
$!(delay spawning anymore until this one concludes)
$ WB +e +s -n 100 -c 5 http://'p1'/ht_root/*.*
$ SPAWN/NOWAIT WB +e +s +n -n 100 -c 1 http://'p1'/ht_root/exercise/16k.txt
$ SPAWN/NOWAIT WB +e +s -n 10 -c 1 http://'p1'/cgi-bin/doesnt-exist
$ SPAWN/NOWAIT WB +e +s -k -n 50 -c 2 http://'p1'/cgi-bin/conan/search
$!(delay spawning anymore until this one concludes)
$ WB +e +s -n 50 -c 2 http://'p1'/ht_root/src/httpd/*.*
$!(etc.)
23.15 - Server Workout (obsolete)
WWWRKOUT Obsolete with 8.0
As the WASD Bench :^) Utility now provides much of the stress-test functionality the WWWRKOUT utility supplied with earlier version of WASD has been declared obsolete.