Using m4 to write
1. Some limitations of HTML
It's amazing how easy it is to write simple HTML pages - and
the availability of WYSIWYG HTML editors like
NETSCAPE GOLD lulls one into a mood of "don't
worry, be happy". However, managing multiple,
interrelated pages of HTML rapidly gets very, very
difficult. I recently had a slightly complex set of pages
to put together and it started me thinking - "there has
to be an easier way".
I immediately turned to the WWW and looked up all sorts of
tools - but quite honestly I was rather disappointed. Mostly,
they were what I would call Typing Aids - instead of
having to remember arcane incantations like
href="link">text</a>, you are given a button or a
magic keychord like ALT-CTRL-j which remembers the syntax and
does all that nasty typing for you.
Linux to the rescue! HTML is built as ordinary text
files and therefore the normal Linux text management
tools can be used. This includes the revision control tools
such as RCS and the text manipulation tools like
awk, perl, etc. These offer significant help in
version control and managing development by multiple users
as well as in automating the process of extracting from a
database and displaying the results (the classic
|sort |awk" pipeline).
The use of these tools with HTML is documented elsewhere, e.g. see Jim Weinrich's article in Linux Journal Issue 36, April 1997, "Using Perl to Check Web Links" which I'd highly recommend as yet another way to really flex those Linux muscles when writing HTML.
What I will cover here is a little work I've done recently with using m4 in maintaining HTML. The ideas can probably be extended to the more general SGML case very easily.
2. Using m4
I decided to use m4 after looking at various other
pre-processors including cpp, the C
front-end. While cpp is perhaps a little too
C-specific to be very useful with HTML, m4 is a
very generic and clean macro expansion program - and it's
available under most Unices including Linux.
Instead of editing *.html files, I create *.m4 files with my favourite text editor. These look something like this:
m4_include(stdlib.m4) _HEADER(`This is my header') <P>This is some plain text<P> _HEAD1(`This is a main heading') <P>This is some more plain text<P> _TRAILER
The format is simple - just HTML code but you can now include files and add macros rather like in C. I use a convention that my new macros are in capitals and start with "_" to make them stand out from HTML language and to avoid name-space collisions.
The m4 file is then processed as follows to create an .html file e.g.
m4 -P <file.m4 >file.html
This is especially easy if you create a "makefile" to automate this in the usual way. Something like:
.SUFFIXES: .m4 .html .m4.html: m4 -P $*.m4 >$*.html default: index.html *.html: stdlib.m4 all: default PROJECT1 PROJECT2 PROJECT1: (cd project2; make all) PROJECT2: (cd project2; make all)
The most useful commands in m4 include the following which are very similar to the cpp equivalents (shown in brackets):
Some other commands which are useful are:
3. Examples of m4 macros
3.1 Sharing HTML elements across several page
In many "nests" of HTML pages, each page shares elements such as a button bar like this:
[Home] [Next] [Prev] [Index]
This is fairly easy to create in each page - the trouble is that if you make a change in the "standard" button-bar then you then have the tedious job of finding each occurance of it in every file and then manually make the changes.
With m4 we can more easily do this by putting the
shared elements into an
m4_include statement, just like
While I'm at it, I might as well also automate the naming of
pages, perhaps by putting the following into an include
m4_define(`_BUTTON_BAR', <a href="homepage.html">[Home]</a> <a href="$1">[Next]</a> <a href="$2">[Prev]</a> <a href="indexpage.html">[Index]</a>)
and then in the document itself:
m4_include button_bar.m4 _BUTTON_BAR(`page_after_this.html', `page_before_this.html')
The $1 and $2 parameters in the macro definition are replaced by the strings in the macro call.
3.2 Managing HTML elements that often change
It is very troublesome to have items change in multiple HTML
pages. For example, if your email address changes then you
will need to change all references to the new
address. Instead, with m4 you can do something like
this in your
and then just put
_EMAIL_ADDRESS in your
A more substantial example comes from building strings up
with multiple components, any of which may change as the
page is developed. If, like me, you develop on one machine,
test out the page and then upload to another machine with a
totally different address then you could use the
m4_ifdef command in your
stdlib.m4 file (just
#ifdef command in cpp):
m4_define(`_LOCAL') . . m4_define(`_HOMEPAGE', m4_ifdef(`_LOCAL', `//127.0.0.1/~YourAccount', `http://ISP.com/~YourAccount')) m4_define(`_PLUG', `<A REF="http://www.ssc.com/linux/"> <IMG SRC="_HOMEPAGE/gif/powered.gif" ALT="[Linux Information]"> </A>')
Note the careful use of quotes to prevent the variable
_LOCAL from being expanded.
takes on different values according to whether the variable
_LOCAL is defined or not. This can then ripple
through the entire project as you make the pages.
In this example,
_PLUG is a macro to advertise
Linux. When you are testing your pages, you use the
local version of
_HOMEPAGE. When you are ready to
upload, you can remove or comment out the
definition like this:
... and then re-make.
3.3 Creating new text styles
Styles built into HTML include things like
<EM> for emphasis and
<CITE> for citations. With m4 you can define your own, new styles like this:
If, later, you decide you prefer
<EM> it is a simple matter to change the
definition and then every
_MYQUOTE paragraph falls
into line with a quick
The classic guides to good HTML writing say things like "It
is strongly recommended that you employ the logical styles
<EM>...</EM> rather than the physical
styles such as
<I>...</I> in your documents."
Curiously, the WYSIWYG editors for HTML generate purely
physical styles. Using these m4 styles may be a good
way to keep on using logical styles.
3.4 Typing and mnemonic aids
I don't depend on WYSIWYG editing (having been brought up on troff) but all the same I'm not averse to using help where it's available. There is a choice (and maybe it's a fine line) to be made between:
<BLOCKQUOTE><PRE><CODE>Some code you want to display. </CODE></PRE></BLOCKQUOTE>
_CODE(Some code you want to display.)
In this case, you would define
_CODE like this:
Which version you prefer is a matter of taste and convenience although the m4 macro certainly saves some typing and ensures that HTML codes are not interleaved. Another example I like to use (I can never remember the syntax for links) is:
m4_define(`_LINK', <a href="$1">$2</a>)
<a href="URL_TO_SOMEWHERE">Click here to get to SOMEWHERE </a>
_LINK(`URL_TO_SOMEWHERE', `Click here to get to SOMEWHERE')
3.5 Automatic numbering
m4 has a simple arithmetic facility with two operators
m4_decr which act as you might
expect - this can be used to create automatic numbering,
perhaps for headings, e.g.:
m4_define(_CARDINAL,0) m4_define(_H, `m4_define(`_CARDINAL', m4_incr(_CARDINAL))<H2>_CARDINAL.0 $1</H2>') _H(First Heading) _H(Second Heading)
<H2>1.0 First Heading</H2> <H2>2.0 Second Heading</H2>
3.6 Automatic date stamping
For simple, datestamping of HTML pages I use the
m4_esyscmd command to maintain an automatic
timestamp on every page:
This page was updated on m4_esyscmd(date)
This page was last updated on Fri May 9 10:35:03 HKT 1997
Of course, you could also use the date, revision and other
facilities of revision control systems like RCS or
$Date: 2002/10/09 22:24:22 $.
3.7 Generating Tables of Contents
Using m4 allows you to define commonly repeated phrases and use them consistently - I hate repeating myself because I am lazy and because I make mistakes, so I find this feature absolutely key.
A good example of the power of m4 is in building a table of contents in a big page (like this one). This involves repeating the heading title in the table of contents and then in the text itself. This is tedious and error-prone especially when you change the titles. There are specialised tools for generating tables of contents from HTML pages but the simple facility provided by m4 is irresistable to me.
3.7.1 Simple to understand TOC
The following example is a fairly simple-minded Table of
Contents generator. First, create some useful macros in
m4_define(`_LINK_TO_LABEL', <A HREF="#$1">$1</A>) m4_define(`_SECTION_HEADER', <A NAME="$1"><H2>$1</H2></A>)
Then define all the section headings in a table at the start of the page body:
m4_define(`_DIFFICULTIES', `The difficulties of HTML') m4_define(`_USING_M4', `Using <EM>m4</EM>') m4_define(`_SHARING', `Sharing HTML Elements Across Several Pages')
Then build the table:
<UL><P> <LI> _LINK_TO_LABEL(_DIFFICULTIES) <LI> _LINK_TO_LABEL(_USING_M4) <LI> _LINK_TO_LABEL(_SHARING) <UL>
Finally, write the text:
. . _SECTION_HEADER(_DIFFICULTIES) . .
The advantages of this approach are that if you change your headings you only need to change them in one place and the table of contents is automatically regenerated; also the links are guaranteed to work.
Hopefully, that simple version was fairly easy to understand.
3.7.2 Simple to use TOC
The Table of Contents generator that I normally use is a bit more complex and will require a little more study, but is much easier to use. It not only builds the Table, but it also automatically numbers the headings on the fly - up to 4 levels of numbering (e.g. section 220.127.116.11 - although this can be easily extended). It is very simple to use as follows:
_H1(`Heading for level 1')or
_H2(`Heading for level 2')as appropriate.
The code for these macros is a little complex, so hold your breath:
m4_define(_Start_TOC,`<UL><P>m4_divert(-1) m4_define(`_H1_num',0) m4_define(`_H2_num',0) m4_define(`_H3_num',0) m4_define(`_H4_num',0) m4_divert(1)') m4_define(_H1, `m4_divert(-1) m4_define(`_H1_num',m4_incr(_H1_num)) m4_define(`_H2_num',0) m4_define(`_H3_num',0) m4_define(`_H4_num',0) m4_define(`_TOC_label',`_H1_num. $1') m4_divert(0)<LI><A HREF="#_TOC_label">_TOC_label</A> m4_divert(1)<A NAME="_TOC_label"> <H2>_TOC_label</H2></A>') . . [definitions for _H2, _H3 and _H4 are similar and are in the downloadable version of stdlib.m4] . . m4_define(_End_TOC,`m4_divert(0)</UL><P>')
One restriction is that you should not use diversions within your text, unless you preserve the diversion to file 1 used by this TOC generator.
3.8 Simple tables
Other than Tables of Contents, many browsers support tabular information. Here are some funky macros as a short cut to producing these tables. First, an example of their use:
<CENTER> _Start_Table(BORDER=5) _Table_Hdr(,Apples, Oranges, Lemons) _Table_Row(England,100,250,300) _Table_Row(France,200,500,100) _Table_Row(Germany,500,50,90) _Table_Row(Spain,,23,2444) _Table_Row(Denmark,,,20) _End_Table </CENTER>
...and now the code. Note that this example utilises m4's ability to recurse:
m4_dnl _Start_Table(Columns,TABLE parameters) m4_dnl defaults are BORDER=1 CELLPADDING="1" CELLSPACING="1" m4_dnl WIDTH="n" pixels or "n%" of screen width m4_define(_Start_Table,`<TABLE $1>') m4_define(`_Table_Hdr_Item', `<th>$1</th> m4_ifelse($#,1,,`_Table_Hdr_Item(m4_shift($@))')') m4_define(`_Table_Row_Item', `<td>$1</td> m4_ifelse($#,1,,`_Table_Row_Item(m4_shift($@))')') m4_define(`_Table_Hdr',`<tr>_Table_Hdr_Item($@)</tr>') m4_define(`_Table_Row',`<tr>_Table_Row_Item($@)</tr>') m4_define(`_End_Table',</TABLE>)
4. m4 gotchas
Unfortunately, m4 is not unremitting sweetness and light - it needs some taming and a little time spent on familiarisation will pay dividends. Definitive documentation is available (for example in emacs' info documentation system) but, without being a complete tutorial, here are a few tips based on my fiddling about with the thing.
4.1 Gotcha 1 - quotes
m4's quotation characters are the grave accent ` which starts the quote, and the acute accent ' which ends it. It may help to put all arguments to macros in quotes, e.g.
_HEAD1(`This is a heading')
The main reason for this is in case there are commas in an
argument to a macro - m4 uses commas to separate macro
_CODE(foo, bar) would print the
foo but not the
bar') works properly.
This becomes a little complicated when you nest macro calls as in the m4 source code for the examples in this paper - but that is rather an extreme case and normally you would not have to stoop to that level.
4.2 Gotcha 2 - Word swallowing
The worst problem with m4 is that some versions of it "swallow" key words that it recognises, such as "include", "format", "divert", "file", "gnu", "line", "regexp", "shift", "unix", "builtin" and "define". You can protect these words by putting them in m4 quotes, for example:
Smart people `include' Linux in their list of computer essentials.
The trouble is, this is a royal pain to do - and you're likely to forget which words need protecting.
Another, safer way to protect keywords (my preference) is to
invoke m4 with the
--prefix-builtins option. Then, all builtin macro
names are modified so they all start with the prefix
m4_ and ordinary words are left alone. For example,
using this option, one should write
define (as shown in the examples in this
The only trouble is that not all versions of m4 support this option - notably some PC versions under M$-DOS. Maybe that's just another reason to steer clear of hack code on M$-DOS and stay with Linux!
4.3 Gotcha 3 - Comments
Comments in m4 are introduced with the # character -
everything from the # to the end of the line is ignored by
m4 and simply passed unchanged to the output. If you
want to use # in the HTML page then you would need to quote it
like this - `#'. Another option (my preference) is to
change the m4 comment character to something exotic
m4_changecom(`[[[[') and not have to
worry about `#' symbols in your text.
If you want to use comments in the m4 file which do not
appear in the final HTML file, then the macro
m4_dnl (dnl = Delete to New Line) is for you. This suppresses everything
until the next newline.
m4_define(_NEWMACRO, `foo bar') m4_dnl This is a comment
Yet another way to have source code ignored is the
m4_divert command. The main purpose of
m4_divert is to save text in a temporary buffer for
inclusion in the file later on - for example, in building a
table of contents or index. However, if you divert to "-1"
it just goes to limbo-land. This is useful for getting rid
of the whitespace generated by the
m4_divert(-1) diversion on m4_define(this ...) m4_define(that ...) m4_divert diversion turned off
4.4 Gotcha 4 - Debugging
Another tip for when things go wrong is to increase the amount of error diagnostics that m4 emits. The easiest way to do this is to add the following to your m4 file as debugging commands:
m4_debugmode(e) m4_traceon . . buggy lines . . m4_traceoff
"ah ha!", I hear you say. "HTML 3.0 already has an include
statement". Yes it has, and it looks like this:
<!--#include file="junk.html" -->
The problem is that:
There are several other features of m4 that I have not
yet exploited in my HTML ramblings so far, such as regular
expressions and doubtless many others. It might be
interesting to create a "standard"
general use with nice macros for general text processing and
HTML functions. By all means download my version of
stdlib.m4 as a base for your own hacking. I would be
interested in hearing of useful macros and if there is
enough interest, maybe a Mini-HOWTO could evolve from this
There are many additional advantages in using Linux to develop HTML pages, far beyond the simple assistance given by the typical Typing Aids and WYSIWYG tools.
Certainly, this little hacker will go on using m4 until HTML catches up - I will then do my last make and drop back to using pure HTML.
I hope you enjoy these little tricks and encourage you to contribute your own. Happy hacking!
6. Files to download
You can get the HTML and the m4 source code for this
article here (for the sake of completeness, they're
copylefted under GPL 2):
using_m4.html :this file using_m4.m4 :m4 source stdlib.m4 :Include file makefile