The Answer Gang

The Answer Gang
By Jim Dennis, Ben Okopnik, Dan Wilder, Breen, Chris, and... (meet the Gang) ... the Editors of Linux Gazette... and You!

Contents:

¶: Greetings From Heather Stern

Backing up with tar

Hunting for new desktop hardware

Software suspend troubles

Oh-oh. This isn't looking good for the moment...

Greetings from Heather Stern

Greetings, gentle readers, and welcome once more to the world of the Answer Gang. TAG, we're it

I have to confess that I wondered to myself, what should I babble about this time? As I look at the back issues, I notice some interesting statistics... apparently, I shouldn't feel surprised that things here at Linux Gazette are a bit hectic. That's consistent with all our past Februaries. Ain't tradition grand?

February here where I sit, is often considered the month of romance. I have to admit... I (heart) Linux Gazette.

I mean, it's not like I will be buying it chocolates or wondering what its favorite cologne is. But I put a bunch of work in every month - I really like knowing you people out there are reading (hint, hint; tell us what you like in this stuff)! I love seeing what sort of curious troubles are out there to solve - the most curious, the kinds of things whose answers change over time. And it's always worth seeing what cool answers are out there.

You don't have to actually be a member of the Answer Gang to send us juicy answers, either. An honorary Answerbubble to folks who send us their good stuff - or cc us on the the juicy tip they are sending to someone anyway. When we got the Gang together, the heading mentioned a few of the active posters... and you. That hasn't changed. It's teamwork that makes this all happen, and I'm proud to be here. Thomas and I have teamed up to bring you the juiciest threads this month, and I hope they make your time with Linux this month - "just a little more fun!"

Lightweight, (Almost) Crypto-Free Remote System Operation

By Ray Ingles

"There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies, and the other is to make it so complicated that there are no obvious deficiencies." - C.A.R. Hoare

"Sure I'm paranoid, but am I paranoid ENOUGH?" - Unknown

Introduction

System administrators frequently want to be able to work on the machines they run even when they are far away from them. There are secure tools that allow full remote shell access, like ssh and lsh, but due to their complexity they have suffered critical exploits from time to time. In addition, their overhead can be excessive for some purposes. Fortunately, other options are available that can be used alone or can be combined with remote shells to create a more secure overall system.

Overview

Maybe the pager has just gone off when you're home in bed, and the boss wants you to fix the broken database now. Or perhaps you're out for lunch and someone calls to tell you the mailserver has been cracked and is currently spamming the world, and you need to bring it down fast. Possibly you've checked and your Web server has wedged itself and needs to be restarted. Or suppose you're just on vacation and find you want to update your home Web site with some new photos. In all these cases, you'd like to do something to the machine over the Internet without having to actually sit in front of it - things you don't want just anybody to be able to do.

The Problem

Tools like ssh and lsh are great for allowing secure remote access to your system. They offer essentially full, flexible remote control of a machine, in an encrypted and authenticated manner. But they are complex pieces of software; there's no way to do what they do without being complex. And with complexity comes bugs. SSH and lsh, and related tools like Webmin, have all had serious flaws that would allow an attacker to get full control over your system. Leaving them available all the time is a risk - sometimes it's necessary, but it's still a risk. And in some cases, you'd like to be able to tell the machine to do something, but it's not even attached to the network on a regular basis.

Some Solutions

It would be nice to enable remote shell access only when necessary. And perhaps (for something like shutting down a mail server) you don't even need a full shell, just a way to fire off a script remotely. Of course, the problem then becomes, how do you know that the alternative software is any more secure than ssh itself? Various people have worked on this problem in the past, and several potential solutions are available, ranging from the simple and venerable to the new and exotic.

Xringd uses a modem to control a machine remotely. Mail filters can be used to trigger actions based on special messages. Some solutions (like 'port knocking' and 'Net::Pcap') use the network, but without requiring even a single open port. Lando runs commands over a network, using username and password. Most recently, a program specifically for secure remote execution called Ostiary has been developed.

The Options

Old School Remote Operation - Xringd

The eXtended Ring Daemon, or "Xringd", uses a modem to monitor rings on a phone line. It counts the number of rings, and the time between them. If a 'sequence' matches one of the ones that it has been set up to detect, Xringd will run an associated command.

This is very nice from a security perspective. Since it uses no network connection at all, it's entirely immune to network attacks like buffer overflows. It can be used even when a network connection is unavailable (it's often used to cause a computer to initiate a dialup connection). The only 'client' you need is a phone. If you use it to start up ssh on demand, then the attacker needs to know the right phone number and the right ring pattern - it's quite hard to sniff that kind of thing remotely. It's also highly resistant to a man in the middle attack. (If you have to worry about someone rerouting your phone calls, you're in more trouble than Xringd can save you from.)

There are some practical issues that may make this unattractive in some circumstances. You need a modem and a telephone line to the server. (Fortunately, you don't need a fast modem at all; even a 1200 baud one will do nicely, but some servers are not placed close to a telephone jack.) Also, things like answering machines or voicemail (or even other people answering the telephone) can interfere with Xringd. If you give the server a dedicated line, you can avoid these problems, but that can be costly.

Finally, note that the rings you hear when making a call are not necessarily synchronized with the ring signals actually sent to the telephone. In most circumstances, they are close enough, but reliability can be an issue at times.

Procmail And Other Mail Filters

Most of the mail filtering programs have a way to invoke scripts when mail matching a pattern is received (in the simplest case, mail to a particular address). Assuming the server is running an SMTP daemon, this can be a nice way of triggering actions remotely. Technically, one could even send a shell script to be run, and have it e-mail the results back to you, giving you the equivalent of a very slow remote shell. The only client needed is an e-mail program, or even a webmail account.

The first problem is that if the box you want to talk to doesn't accept e-mail, this obviously won't work. (Adding an entire mail server, with the attendant risks of bugs, spam load, etc., just for remote execution doesn't make a lot of sense.) Some machines only periodically collect e-mail from a primary server, so there can be a substantial delay between when a command is sent and when it is acted upon.

Furthermore, if you don't encrypt the traffic in some way (or at least sign it with PGP), then anyone sniffing traffic between you and your server may be able to take advantage of the same channel to do mischief, or perform a man-in-the-middle-attack. (E-mail traffic is notoriously easy to falsify; hence the avalanche of spam these days.)

CVTSA, or "ClairVoyanT SysAdmin", is a system designed specifically for running commands through e-mail. It has some support for using passwords, but does not (currently) encrypt them in transit, so a sniffer could capture them and use them again.

Of course, if the only things you want to do with this type of system are emergency shutdowns and other such (hopefully rare) crisis management, then even an unencrypted channel might work. However, you'll need to change the 'magic trigger pattern' each time after you use it, or you take the risk that an attacker might capture it and 'replay' it at an inconvenient time.

Port Knocking

With port knocking, a daemon monitors firewall logs, looking for particular sequences of connection attempts to particular (closed) ports. When it sees a sequence it recognizes, it runs the associated command. This isn't terribly bandwidth efficient, but it has some nice properties. First, it's hard to tell if a server is listening for port knocks. Second (and most important), it's awfully hard to crack a closed port. (Linksys routers have had a simple version of this for a while, BTW, that they call port triggering.)

However, a clever attacker with a sniffer could notice this traffic, and duplicate it for their own use. More complicated encodings could express something like a PGP signature (indeed, in theory one could create an entire network protocol based on port knocks), but things rapidly become difficult to work with. As with 'mail filtering' solutions, one can either use it sparingly in emergencies, or move to real cryptography.

It's also important to realize that this system is critically dependent on the probe packets actually being delivered, and delivered in the order that they were sent. This is not guaranteed on the Internet. What's more, depending on where you're at (e.g., an Internet cafe or behind a business firewall), you might not be allowed to connect out to arbitrary ports. The more complex you make the 'knocks', the less reliable the system will be.

Also, notice that at least one entire IP packet (28 bytes or so minimum) is used to transmit roughly one bit of information. In terms of network efficiency, it's almost hideous. For a simple 'open up ssh' message, it's not a consideration, but actually adding cryptographic security to this system could use up a decent chunk of the available bandwidth.

Finally, this increases the CPU load for each entry in the firewall log. Depending on how detailed the logs are, and how fast and busy the network is, this can be a significant drain on resources.

'Sniffer' Triggers - libpcap and friends

Another interesting approach is to use Net::Pcap or other network capturing software to look for specific packets on the network (e.g., DNS requests) and examine them for particular data (e.g., a particular address). If found, it can enable ssh temporarily, or perform other actions.

One potential benefit of this approach is that a computer doesn't have to have an address on a network in order to monitor traffic on that network. You can set the card to 'promiscuous mode' and examine all the traffic on the wire. (It's very hard to hack a machine you don't even know is there.) Once the 'trigger' is spotted, the sniffer can use other means (a separate network, a serial link, even Xringd) to open up SSH on a target machine. Of course, you can also simply run the sniffer directly on the target.

Again, a clever attacker with their own sniffer may be able to detect the unusual activity and correlate it. To make this system truly secure, you would need more complex encoding/encryption of the 'trigger' traffic.

Additionally, the CPU load for this solution can be even worse than for 'port knocking' systems. A 'port knocking' daemon monitors firewall logs, which can have variable levels of detail. By necessity, a 'sniffer' solution must examine every packet on the network segment, which can be a substantial task for a busy gigabit line.

Lando - Simple & Flexible

Lando allows a user to run a preconfigured set of commands remotely, using passwords, and even allowing the user to supply arguments to them. While it currently has only a Windows client, and passwords are sent in the clear (making it suitable only for use on a trusted local network, or perhaps on a VPN), it can be very useful for, e.g. operating a local firewall box without going to the trouble of logging in.

Okay, A Little Crypto - Ostiary

All of the above solutions have their advantages, but each has some practical issues that can make them unsuitable for particular applications. Ostiary was designed to be a secure alternative that uses minimal resources. It tackles this problem with what might be termed "aggressive simplicity". It does require an active connection to the network (unlike Xringd and sniffing), but allows for much better default security with very low CPU, RAM, disk, and network bandwidth requirements.

An Ostiary server has one open port that it listens on. When someone connects, the server sends a random fixed length 'salt' message 16 bytes in size - the size of an MD5 hash. It then waits (with a timeout) for a reply from the client. It reads (at most) 16 bytes of reply, and closes the connection.

Ostiary has a list of commands to run, with associated passwords. It runs through the list, and hashes these passwords with the 'salt' it sent to the client. If one of these hashes matches the reply from the client, the associated command is run. (One final touch is that a record is kept of connections, and clients with too many failed attempts are 'locked out', and all subsequent communication from them is ignored.)

A detailed security analysis is available, but a few things about this system should be clear. With a protocol this simple, the chances for dangerous bugs are drastically reduced. Using fixed-length messages essentially eliminates the chances of a buffer overflow or other memory error. (Indeed, Ostiary does no dynamic memory allocation of any kind - everything is stored in static, fixed-size data structures.) Replay and man-in-the-middle attacks are also effectively useless. Ostiary limits how fast it accepts connections, enforcing low CPU and network usage. (The first production Ostiary server was a 16MHz 68030 machine.) Client requirements are even lower: Clients are available for Palm Pilots and even Windows.

Unlike a procmail-based solution, where you can put arbitrary commands (with arguments) in the message, Ostiary can only run the fixed set of commands you have preconfigured. The only argument it supplies to the commands is the IP address of the client that initiated the command. It requires an active network connection (unlike Xringd) and an open port (unlike port knocking or sniffing), which may entail configuring a firewall to open a new port. (Although one could run Ostiary on, say, port 22, and upon receipt of the correct command, it could terminate itself and spawn sshd...)

Since Ostiary uses TCP, it is as reliable as the network it uses to communicate. Problems with miscounted phone rings (a la Xringd) or randomly dropped packets (a la port knocking) are not a concern.

Summary

The following table summarizes the pros and cons of the various systems outlined above. "Replay" and "Man-in-the-middle" indicate if the default system is vulnerable to the corresponding attacks. "Command arguments" indicates if the system can run arbitrary commands with arguments. "CPU load" indicates that CPU time can be a significant consideration. "Special client" indicates that a specific client program is needed to work with that system.

System	Xringd	Mail filter	Port knocking	Sniffers	Lando	Ostiary
Network Required?		Yes	Yes	Yes	Yes	Yes
Port Required?		Yes			Yes	Yes
Modem Required?	Yes
Replay?		Yes	Yes	Yes	Yes
Man-in-the-middle?		Yes	Yes	Yes	Yes
Command arguments?		Yes			Yes
CPU load?		Sometimes	Yes	Yes
Special client?			Sometimes	Sometimes	Yes	Yes

None of these approaches is right for everyone. But all of them can be used to make attacks at least more inconvenient, and in many cases far more difficult. Remember, though, to analyze their pros and cons relative to your specific situation. Also remember that true security is a process, not a goal - you can never just install some software and be done thinking about it.

Ray Ingles has been involved with Linux since 1995. In addition to being an active member of the Metro Detroit Linux User's Group, he has made minor contributions to the UPS HOWTO and the Linux Joystick Driver.

XMLTV

By Bill Lovett

Where do you go to find out what's on TV? The usual suspects might include a newspaper, a recent issue of TV Guide magazine, a favorite Web site, or your nearest TiVo, ReplayTV, or other PVR. But don't forget to add Linux to the top of that list. You can let the machine do the dirty work and bring the listings to you. XMLTV, a short bash script, and a cron job are all you need to get started.

Installation

First things first: getting the program installed. XMLTV is a suite of Perl scripts and can be downloaded from membled.com/work/apps/xmltv. There are releases for Unix-like and Windows environments, but, for obvious reasons, we'll focus on the former. If you're installing from source, it's the usual routine:

% perl Makefile.PL
% make
% make test
% make install 

If you're on Debian, it's all just an apt-get away (apt-cache search xmltv). Links to packages for OS X, Red Hat 8, and Red Hat 9 are available from the project's homepage.

Configuration

Before XMLTV can be useful, it needs to know where in the world you are. XMLTV is international— it can fetch TV listings for Canada and the United States, the United Kingdom, Austria and Germany, New Zealand, Finland, Italy, Spain, the Netherlands, Denmark, and Hungary. (Belgium and France are in the works.) The scripts that collect listings for a particular country are referred to as grabbers, and you'll find them on the command line under tv_grab_*. We'll use the U.S. grabber, tv_grab_na.

When you first run the grabber, do so with the --configure option. This starts a question-and-answer session in which you and the grabber get a little bit better acquainted, as far as your Zip code, TV service provider, and channels you want to ignore are concerned. The results of the script are written to ~/.xmltv/tv_grab_na.conf, and can be easily edited by hand.

At this point, XMLTV is ready to do your bidding. Do a man tv_grab_na to learn about all the available options. For now, just two will suffice:

% tv_grab_na --days 1 --output /tmp/tv.xml

XMLTV's file format doesn't quite make for friendly reading, unless you enjoy reading raw markup. A few more scripts from the suite can fix that. tv_sort sorts the contents of an xmltv file date. tv_grep lets you weed out some of the obvious cruft in the listings. Here's how I run it:

% tv_sort --output /tmp/tv_sorted.xml /tmp/tv.xml
% tv_grep --output /tmp/tv_grepped.xml --ignore-case --not --category Children \
          --not --category Sports --not --title "Paid Programming" \
          --not --title "Local Origination" \
          --on-after now /tmp/tv_sorted.xml

At this point, we've still got an XML file. Converters to the rescue! tv_to_text is one of the tools that can help us go from XML to something else. (Other possibilities include LaTeX, HTML and PDF. Check the readme to see what's currently available.) After running something like this:

% tv_to_text --output /tmp/tv.txt /tmp/tv_grepped.xml

21:00--21:30    Spy School      38
21:00--21:30    Designing for the Sexes // European Kitchen     64
21:00--21:30    Chappelle's Show        67
21:00--21:30    The Real World // Las Vegas     71
21:00--22:00    Law & Order: Special Victims Unit // Guilt      44
21:00--22:00    Wild Card // Auntie Venom       45
21:00--22:00    Cold Case Files // The Accidental Killer; Little Sister Lost    57
21:00--22:00    America's Most Wanted: America Fights Back // Top Ten Most Wanted Fugitives     5
21:00--22:00    The FBI Files // The Price of Greed     60
21:00--22:00    Trading Spaces // Nashville: Murphywood Crossing        61
21:00--22:00    Great Chowder Cook-Off  63
21:00--22:00    Ends of the Earth // Secrets of the Holy Land   65
21:00--22:00    The E! True Hollywood Story // The Hilton Sisters       68
...

Delivery

If we stopped at this point we'd have used several of XMLTV's abilities but hardly anything else. We'd also be running low on convenience and automation. Fortunately, we can wrap all the commands we've seen so far into a shell script, and have it e-mail us the final results. mail can take care of, well, the mailing:

% mail -s "Today's TV listings from XMLTV" user@localhost < /tmp/tv.txt

#!/bin/sh

# Grab today's listings:
tv_grab_na --days 1 --output /tmp/tv.xml

# Sort
tv_sort --output /tmp/tv_sorted.xml /tmp/tv.xml

# Grep
tv_grep --output /tmp/tv_grepped.xml --ignore-case --not --category Children \
--not --category Sports --not --title "Paid Programming" \
--not --title "Local Origination" \
--on-after now /tmp/tv_sorted.xml

# Convert To Text
tv_to_text --output /tmp/tv.txt /tmp/tv_grepped.xml

# Email
mail -s "Today's TV listings from XMLTV" user@localhost < /tmp/tv.txt

More importantly, you've got a foundation to build on. What we've covered is just the beginning— beyond the command-line scripts, a GUI client is also available. Of course, there are plenty more things you could do from the command line, such as:

1. Pull in data from imdb.com via tv_imdb
2. Split the listings into separate files for each day and channel via tv_split
3. Transform the XML with your own XSLT stylesheet.
4. Only send e-mail if certain keywords are found

Bill Lovett is a Web developer in New York City. He's one of those PHP/MySQL types. And he has this weird thing about running Linux on old machines that by all rights should have been trashed years ago. Read more about Bill and his Open Source projects at www.ilovett.com

Let's Build a Cool Linux Toy

By Pramode C.E

Many of us make a living out of Linux - but, if somebody asks us why we are so crazy about it, one common answer would be `fun'. Playing with Linux is lots of fun - with the added benefit that, most of the time, you end up learning a lot. Recently, I happened to come across a nice book which tries to emphasize the `fun' aspect of Linux - it describes several small `projects' (a jukebox, a picture frame, etc.) that a moderately experienced Linux user may be able to implement on her own. One of the projects involved interfacing with a temperature-sensing element and putting up the temperature value on a Web page (or including it in your email signature - and any other crazy stuff which you can imagine!). The only trouble was that, in the part of the world where I live, walking up to an electronics store and asking for an integrated, 1-wire temperature-sensing element is more likely to yield a hard stare than anything else. Smart sensors that can be directly interfaced to the PC with the minimum of fuss are seldom available off-the-shelf - you will mostly have to `roll your own' - which adds to the fun and excitement. With a low-cost general purpose microcontroller like the PIC16F628, bits and pieces of cheap, commonly available electronics components, and LOTS of code, you can build many interesting `toys' and hook them up to your Linux machine - a really great learning experience for the hardware hacker who wants to learn Linux, or the Linux hacker who wants to learn a bit of hardware. This article describes how I went about building my temperature-sensing project - amateur Linux/hardware hackers might find some of the ideas useful when they start building things on their own.

Get a PIC micro, and set it up to work with Linux

This is the first step. Microchip PIC controllers are commonly available. If you are like me, working with a soldering iron for more than 10 minutes would drive you crazy - so you have to choose the right kind of PIC - the one that can be programmed with the simplest possible circuit (connected to the PC parallel port), preferably with a 5V supply. Look no further than the PIC16F628. This is a cool device that has lots of peripherals (except the ADC - but then, we can roll our own crude analog-to-digital converter with the comparator and pulse width modulation facilities offered by the PIC) and supports a `Low-Voltage Programming Mode'. I found a nice little circuit (the simplest circuit, and one that works perfectly, out of the dozens I have seen on the Net) designed by Jim Paris for a microcontroller programming laboratory at MIT. Here is the circuit:

[diagram]

I assembled the circuit on a breadboard for testing in a few minutes' time.

Jim Paris has designed a program (called `jimpic') for burning machine code onto the flash memory of the microcontroller. It is available for download from here. I wrote a simple assembly language program, converted it into machine code with the help of the `gpasm' assembler available as part of the GNU PIC Utilities Project and burned it onto the micro by running `jimpic' with the `-b' option.

A Quick Introduction to PIC programming

A nice thing about the PIC is that, if you have some background in general microprocessor architecture and assembly language programming, you can become productive with it in just about one or two hours' time. The instruction set is very compact (35 instructions) and sufficient for most simple bit-twiddling tasks. The PIC16F628 packs a decent 224 bytes of data memory with 2K of program (code) memory. The peripherals include general-purpose digital I/O ports, three timers, two analog comparators, on-chip voltage reference module, Universal Synchronous-Asynchronous Receiver Transmitter (for serial communication), and Capture-Compare-PWM module. Special CPU features include a watchdog timer, brown-out detect circuitry, and an internal RC oscillator (so that you won't be needing an external crystal if you aren't too concerned about precise timing).

The general purpose data RAM begins at address 0x20 (the locations below this are Special Function Registers - basically memory mapped I/O ports, control registers etc.). Here is an elementary assembly language program, which simply stores the value 0 into the accumulator (the `W' register, in PIC terminology).

[Listing 1]

(Remove the .txt extension if you download the listing. It's there only to ensure browsers display it properly.)

We will now assemble the file:

gpasm -a inhx8m a.s

The result is an Intel hex format file, which can be given to `jimpic' for burning. Each line of the hex file contains a few bytes of machine code, the address at which the machine code is to be stored (in the flash memory of the microcontroller), some kind of checksum, and some other information. Here is the hex file generated by running `gpasm' over our assembly language program:

:020000000030CE
:02400E00983FD9
:00000001FF

The first line of our program tells the assembler that machine code is to be generated for the PIC16F628. The second line includes a file (available with the `gputils' distribution) that contains lots of symbol definitions. The third line, a __CONFIG directive, tells the assembler what special features of the microcontroller (say, the Watchdog timer) should be enabled/disabled by writing bit patterns to a `magic' `configuration word' within the PIC; _WDT_OFF means we don't want the watchdog to be enabled, _INTRC_OSC_NOCLKOUT means we are going to use the internal oscillator to provide the timing signals necessary for program execution. You will have to refer to the 16F628 datasheet to know more about these configuration bits. The fourth line is the only proper assembly language instruction in the program - it moves the `literal' (constant) value 0 to the `W' register. Note that each line begins with a tab.

Lighting up an LED

Here is a program that lights up an LED connected to the RB0 pin of the microcontroller:

[Listing 2]

PORTB is an eight-bit port - the direction of each pin (i.e., whether the pin is to act as input or output) is controlled by individual bits of the TRISB register - if a TRISB bit is set, the corresponding PORTB pin is input - otherwise it is output. The PIC has the concept of `banked' addresses, which is rather confusing to the beginner. (It's a headache even if you are an `experienced' developer.) You visualize `banks' of special function registers - the STATUS register is the same across all the banks while the TRISB register is available only in bank 1. You are by default in bank 0. To access TRISB, you have to `switch over' to bank 1. This is by setting the RP0 bit of the status register. (When you read microcontroller manuals, you will see that not only are the control registers given special names, even the individual bits are named. Header files available with the development kit for the microcontroller map these symbolic names to the numbers given in the manual, making the life of the assembly programmer a bit easier.) The `bsf' instruction (bit set f - `f' represents the fact that the number that comes as the operand for the instruction represents a memory address or a special function register and not a `literal') takes two operands - the first one being the address of a RAM location or a special function register, and the second, a bit number. The `movwf' instruction copies the contents of the `W' register to the memory location whose address is the operand of the instruction.

Building a `running' circuit

After assembling and burning the above program, we are ready to see it in action. The running circuit can be built in a jiffy - place +5V on the VDD pin of the PIC (pin 14), connect Vss (pin 5) to circuit ground, connect MCLR (pin 4) to +5V through a 2K resistor, connect the LED between RB0 and Gnd with a current limiting resistor of say 1K in series - and that's all. You should see the LED lighting up as soon as you apply power. Your next attempt will be to make the LED blink - for that you will have to read a little bit more about the PIC instruction set - the manual will come in handy at this juncture.

Debugging tips

Here are some things that I have found handy while debugging:

• Check the power supply
• Don't jump into conclusions that the hardware is wrong - you might have misinterpreted the datasheet, your program logic might be wrong, or worse still - the data sheet might be WRONG. Which brings us to the next rule, which is:
• Always read the manufacturers' errata - if they have one. The 16F628 datasheet contains some errors - especially concerning writes to EEPROM data memory and the behavior of the MCLR pin in low-voltage programming mode.
• Don't think the hardware will never malfunction - for example, the PIC might consume larger current when writing to the internal data EEPROM; your battery-powered supply might not be able to deliver the required current, and your program will misbehave. If you have an external crystal, it might not be working properly, and the micro might not be getting its clock.
• Google Groups is your friend - use it wisely. Search the archive; somebody might have experienced the same problem before. Post a message if you feel that your problem is something `original'.

The temperature sensor interfacing project

The LM35 is a commonly available calibrated temperature sensor that converts temperature (in degrees Celsius) to voltage - each degree rise in temperature results in 10mV rise in output voltage. It's a three-pin device - Vcc, Gnd, and voltage output. You can get the datasheet from here. Say the current temperature is 23 degrees Celsius; the voltage output would be 230 millivolts.

The question is, how do you convert this voltage to a digital value? The easiest way would be to use a commodity analog-to-digital converter, and interface it to the printer port. Another solution would be to use a PIC with a built in ADC (say the 12F675). The third would be to use some of the peripherals available in the 16F628, write some code, and build a crude ADC of your own. As I had explored the first two options a lot in the past, I thought of trying out the third one.

Two peripheral features of the PIC are of interest to us here - one is the builtin PWM module (Pulse Width Modulation), which is capable of generating, in, hardware, a continuous stream of digital on-off pulses whose duty cycle can be varied simply by storing certain numbers in specific special function registers. Once the PWM module is initialized to generate a pulse train of a specific duty cycle, it will keep on doing so without any software intervention - our program can do something else.

The PIC is also equipped with two analog comparators, which can be configured in a variety of ways. Let's say we are using just one of the comparators. Two PORTA pins can be programmed to accept voltage levels and transmit them to the Vin+ and Vin- pins of the comparator. The comparator output is high if the Vin+ voltage is greater than the Vin- voltage, and low otherwise. The output can be made available on another PORTA pin, or it can be simply read from a particular bit of the Comparator Control Register, CMCON.

Filtering PWM pulses

The figure shows a PWM pulse (off OV, on +5V) of period T being fed to an RC circuit (R*C >> T). If the on-off periods are equal, the output seen across the capacitor will be a constant DC level of magnitude 2.5V. Electrical engineering text books should give you the reason why it is so - or, if you are not very sure of the math involved (as I am), play with some R and C values until you get the desired effect. Now what if you feed a PWM pulse whose on-time is less than T/2? You will see that the output is again a DC level, but the magnitude has come down proportionately. What if you increase the on-time? Again, the output is a DC level, only thing is the magnitude has increased proportionately. Now you have a cool way to implement a DAC, a digital-to-analog converter. Say you want to generate a voltage of 0.449V. What if you program the PIC so as to generate a PWM pulse train of period 256 microseconds and on-time 128micro. The output voltage would be 5V*(128/256.0) = 2.5V. Now, what if the on-time is 23 micro seconds? The output is 5V*(23.0/256) = 0.449V. (I use Python to do these quick-and-dirty calculations. It's one of my favourite uses of this great language.) The on-time can be altered easily by writing some numbers to two registers, CCPR1L and CCP1CON. A pure digital way to generate analog voltage!

From DAC to ADC

What has all this got to do with converting the LM35 sensor's analog voltage output to a numerical value? Well, a DAC, together with a comparator, builds up an ADC. How? Say the maximum and minimum temperatures at your place of residence can never go above/below 45 degree Celsius and 20 degree Celsius. So the sensor's output will always be between .45V and .2V (remember, 10mV per degree change in temperature). We start generating a PWM signal of period 256 microseconds. The RC-filtered output is fed to Vin+ of the comparator, and the sensor's output is fed to Vin-. Let's say the sensor output is .3V. If the PWM on-period is 23 microseconds, the filtered DC level would be 5*(23.0/256) = 0.449V. So, Vin+ is greater than Vin-, and the comparator output (as obtained from a bit of the CMCON register) is high. Now, we start bringing down the on-time. At a certain point, Vin- will go above Vin+, and the comparator output drops to zero. The magnitude of the on-time at this point is a true representation of the analog value of the sensor output. We communicate this number to a program running on the Linux box through a serial link. You can download the PIC assembly language program that does all these tricks:

[Listing 3]

Instead of performing a `linear' search from the high boundary down to the lower one, we can think of generating a voltage that lies in the middle of this range and comparing it with the sensor output. If the comparator says that the sensor output is higher, we can repeat the same procedure on the upper half. This is the classical `binary search' applied to solve a hardware problem! Horowitz and Hill, in their book The Art of Electronics, have oscilloscope traces of this binary search in action! Computer programmers should always show a good amount of skepticism towards code that claims to do binary search - the algorithm looks deceptively simple - but is in fact not very easy to implement correctly.

Back to Linux

The PIC micro sends the temperature data it has gathered out through a port pin (RB2) in a serial manner - this port pin is directly connected to the receive pin of the PC serial port. What remains is to write a program that will read this data and process it in some manner. Even though the RS-232C serial communication standard defines an `on' voltage to be between -3 and -12V and an `off' to be between +3 and +12, I have been able to get satisfactory results using the 0 and 5V logic outputs from the PIC port pin - if it doesn't work out for you, you will have to place a device like the MAX232 between the PIC port pin and the PC serial port receive pin.

Interfacing with the serial port

Let's look at the simplest way to interface an external circuit to the serial port. (We won't be sending any data out through the PC serial port - that would make the circuit a wee bit more complex.) Pin number 2 of the 9-pin PC serial port connector is the receive pin, 3 the transmit pin, and 5, Ground. Let's say the PIC is sending data out through its RB2 pin at 9600 bits per second, 8N1 (8 data bits, no parity, 1 stop bit) format. The UART that controls the PC serial port should be programmed for this particular baud rate and data format. This can be done by writing magic bit patterns to certain control registers. Once that is done, our program can keep on polling a bit of the UART status register to know whether a new data byte has arrived. Here is the code listing:

[Listing 4]

The program has two disadvantages. One, it is using low-level I/O calls, which, if they are to work properly, should be preceded by an iopl() call. Only the superuser can call iopl() successfully - so the program should run under root privilege. We are wasting CPU time when we keep polling for data in a tight loop; that's another big problem. Both are solved by not directly interacting with the hardware - we can make use of system calls to talk to the serial driver within the Linux kernel - which will do all the low level stuff needed to manage blocking, interrupt driven I/O.

Serial I/O in Python

The Python `termios' module can be used for doing serial comm at a higher level. Isaac Barona Martinez has written a simple wrapper over `termios'. It is called uspp and is available for download from here. Using this module, reading from the serial port is a breeze:

[Listing 5]

from uspp import *

# COM1 is initialized at 9600 baud. The
# default data format is 8N1

s = SerialPort("/dev/ttyS0", None, 9600)
s.flush() # discard unread bytes

print ord(s.read()) # s.read() returns a one-character
                    # string. We convert it into its ascii
                    # value

• Write a simple server that accepts connections over the network and transmits the current temperature
• Write a program that keeps on reading the temperature at say, half-hour intervals. The temperature reading, together with some stupid message like `Oh - it's burning hot here' can be placed at the end of your `.signature' file!
• Another idea would be to use something like the Python `ftplib' to upload the temperature reading to your Web server periodically.

Acknowledgements

Thanks to Christopher Negus and Chuck Wolber for a really cool book!

Thanks to Jim Paris, Ariel Rodriguez and Sheldon Chan for the excellent `jimpic' hardware and software. As I had mentioned earlier in this article, I find it to be the easiest way to get started with PIC programming under Linux. Thanks to Isaac Barona Martinez for uspp, which simplifies serial programming a lot.

Conclusion

There are two excellent documents that describe serial programming under Linux. One is the Serial Programming HOWTO. The other is Serial Programming guide for POSIX operating Systems. The Microchip home page contains lots of application notes, reading which might give you ideas for your next Linux hardware hack - just don't forget to share the fun with LG readers! I can be contacted via my home page at pramode.net.

I am an instructor working for IC Software in Kerala, India. I would have loved becoming an organic chemist, but I do the second best thing possible, which is play with Linux and teach programming!

OCaml, an Introduction

By Jurjen Stellingwerff

Object Caml is an ML type of language. For the non-gurus: it's a functional language that can also be programmed in a non-functional and object-oriented way.

This language is really easy to learn. It's powerful and keeps impressing me with its speed. Programs written in this language are almost always stable by default. No segmentation faults, only occasional unending loops for the programmers that still hang on to program their own loops. It is really not needed to write most loops, since the libraries contain standard functions that are good enough in 99% of the cases. So try to use those functions: It really pays off in terms of stability of your programs, and, unless you have intimate knowledge of the inner works of this language, they tend to be better optimised.

The language can be obtained from the website caml.inria.fr. Here, they provide RPMs for the RedHat 7.2/8.0/9 and Mandrake 8.0 distributions. Also MS Windows binaries are available, but not all Unix library functions will work there, for some mysterious reason. The source tarball does compile flawlessly for me. It just has a somewhat unusual makefile layout:

# ./configure
# make world; make opt; make install

The normal libraries include many usable data-structures like balanced trees, hash tables, and streams. Their version of header files (.mli files) contain all the basic documentation you need, and those are directly converted into HTML and published on the Web in their OCaml manual. This manual is not very usable to study this language, so I'll try to explain here some of the basic language constructions. This is just to give you an impression of the power of this language.

Modules & Functions

Now some real life examples. I wrote a program to help administrating a computer. It is a subset of a normal file finder, but is a command line tool and very fast. It helps locating large, not-recently-used files to be deleted from the system. It crawls through the directory tree and show the contents in different layouts.

Every module in OCaml has its own namespace. Specific definitions can be found by adding the module name, with the first character an upper-case character. You can also change the namespace of the current program to include a total module. Normally, only the standard module 'pervasives.mli' is included in the default namespace. The example program 'show.ml' starts with:

open Basics
open Unix
open Unix.LargeFile

This includes my own set of 'basics' functions and 2 standard libraries: 'Unix' and 'Unix.LargeFile'. A module normally consists of 2 files. The first file for exporting definitions 'module.mli' (like the C .h file), and the second one for actual code (the 'module.ml' file). The program uses the function 'string_sub' that provides a foolproof version of the 'String.sub' standard function (from the string.mli module). The basics.mli file contains the lines:

val string_sub: string -> int -> int -> string
(** Get the sub string from a [string] from position [from] with [length].
This is the same function as String.sub, but it will never raise an exception.
And a negative [from] value is counted from the right side of the string. *)

This gives the definition of this function and the description. There is an automatic documentation generator (ocamldoc) that reads .mli files and writes .html files as basic interface documentation. Normal comments start with (* but the documentation generator only writes comments that start with (** to the .html files. This document contains links to the documentation of the used modules. This documentation is really helpful to start programming ocaml. The .mli files are all included in the distribution, but the complete manual and a book can be downloaded from the Web site caml.inria.fr

The function is followed by its type. It wants 3 parameters and provides a string. Normally we need to write 'Basics.string_sub' to use this function. But after the 'open Basics' instruction just 'string_sub' is enough.

Basic operations and function calls

Now, back to the main program again. The first function is 'gettype'. It will try to return the type of a file. The file type is defined as the part of the filename following the last '.'. When there is no dot, the type is unknown and returned empty.

let gettype file =
try
let pos = String.rindex file '.' in
String.sub file (pos+1) (String.length file-pos-1)
with Not_found -> ""
;;

This function only uses standard functions. First, it catches the Not_found exception in the 'try' 'with Not_found -> ""' code. All other exceptions will be passed to the caller to be handled, and can possibly stop the main program. The local variable pos get is filled with the result of the function rindex. This function is also the reason to catch the exception; otherwise, the main program might stop on the first found file with no '.' in it. Local variables can be declared everywhere inside ocaml with 'let <variable> = <value> in <code>'. After the completion of the given code, the variable is out of scope and will be forgotten. The data will be passed to the garbage collector to be removed from memory. Function calls do normally use brackets. The function call to 'String.sub' gets 3 parameters the string 'file' the integer '(pos+1)' and the integer '(String.length file-pos-1)'. The last parameter calls the function 'String.length' with a single parameter 'file'. So, the functions are eager for their parameters; brackets are needed only when the parameters are filled with calculations.

Also '(+)' and '(-)' are functions of the pervasives module. It is very easy to define your own operators; just add brackets around their definition, and they are ready.

If then else

The next routine 'filesize' in the example code is far longer, but largely introduces sub-functions and 'if <bool-expr> then <expr> else <expr>' statements. This function creates a string from an int64 number for human readable file and directory sizes. The types of parameters are normally not given; they are determined by ocaml through their usage. When something is not clear, the compiler or interpreter will complain about it before executing the code.

let filesize s =
let tostr f =
  if f>9.9 then
    string_of_int (int_of_float (f +. 0.5))
  else
    let res = string_of_float (floor (f *. 10.0 +. 0.5) /. 10.0) in
    if String.length res=2 then
      res ^ "0"
    else 
      res
in
let bytes = Int64.to_float s in
if bytes > 512.0 then
  let kb = bytes /. 1024.0 in
  if kb > 512.0 then
    let mb = kb /. 1024.0 in
    if mb > 512.0 then
      let gb = mb /. 1024.0 in
      tostr gb ^ " Gb"
    else
      tostr mb ^ " Mb"
  else
    tostr kb ^ " kb"
else
  Int64.to_string s
;;

The ocaml standard library has a set of conversion functions. These functions normally follow the form of 'int_of_float' and 'string_of_float'. Specific types like 'Int64' use shorthand notations like 'Int64.to_float'. String concatenations are done with the operation '(^)'. Normally, functions are defined for only one specific type, so there are new sets of arithmetic functions for floats like '(+.)', '(*.)' and '(/.)'. The 'tostr' sub-function has some extra calculation to change something like '5. Gb' into the nicer form of '5.0 Gb'.

List notation and type conversion

The next function, 'converttime', converts a string into a float. OCaml uses floats for date for 2 reasons. The first is to prevent possible Year 2k problems, and can also be used for less than one-second time measurements. The function accepts English acronyms for month names. So let's introduce the list and the pair to create a translation of acronyms into numbers.

let month = [("jan", 0); ("feb", 1); ("mar", 2); ("apr", 3); ("may", 4); ("jun", 5);
             ("jul", 6); ("aug", 7); ("sep", 8); ("oct", 9); ("nov", 10); ("dec", 11)]
;;

This list is totally static, and can be used easily by the standard function List.assoc to convert a string into the corresponding number.

let converttime str =
try
begin match
  if str>"a" && str<"z" then
  ( int_of_string (string_sub str (String.rindex str ' '+1) 99),
    List.assoc (string_sub str 0 3) month,
    1
  )
  else
  ( int_of_string (string_sub str 0 (
      try String.index str '-' with Not_found -> 99
    )),
    ( try let pos=String.index str '-'+1 in
        int_of_string (string_sub str pos (
          try String.index_from str pos '-'-pos with err -> 99
        ))-1
        with err -> 0
    ),
    ( try let pos=String.index str '-'+1 in
        int_of_string (string_sub str (String.index_from str pos '-'+1) 99)
        with err -> 1
    )
  )
with (yr,mn,md) ->
(* print_string ("Last access before: "^
     string_of_int (if yr<50 then yr+2000 else if yr<100 then yr+1900 else yr)^"-"^
     string_of_int (mn+1)^"-"^
     string_of_int md^"\n"); 
*)
  fst (mktime 
  { tm_sec = 0; tm_min = 0; tm_hour = 0;
    tm_mday = md; tm_mon = mn;
    tm_year = if yr<50 then yr+100 else if yr<100 then yr else yr-1900;
    tm_wday = 0; tm_yday = 0; tm_isdst = false
  })
end with err ->
  print_string ("Cannot decipher this date string '" ^ str ^ "'\n"); max_float
;;

The new operation in this function is the 'match <expr> with <template> -> expr'. This is one of the most versatile instructions of ocaml. It can be used to examine the contents of variables and get the needed information out of it. This function creates the triplet (year, month, day-of-month) out of 2 different date notations. To debug this function the 'print_string' instruction is included but commented out to prevent clutter in the output of the program. Normally there is some logging mechanism to make the extra messages optional for the user. The 'print_string' shows the ISO notation of the given date; it creates a 4-digits year and gives a month number with January=1 instead of the internal Unix use of January=0.

This function also shows the use of 'try <expr> with err -> <expr>' that caches every possible exception and fills the variable 'err' with the details of the exception. This function can raise quite a lot of different exceptions, and frankly I am not very interested in the details. The routine just complains to the user about the given date string and gets over it. It returns the maximal possible float to include every filename.

The main standard function is the 'Unix.mktime' function. It wants to get a record filled with numbers about the current time. This function returns a pair with the needed float and a normalized record. With the pervasives function fst returns just the first parameter of the pair.

The ';' before the 'max_float' indicates that the expression results in a float, but the instructions before the ';' are calculated first. This is the first non-functional instruction inside the example code. OCaml is not strictly functional, but has the full power of other functional languages.

Dynamic data structure

Now is the time for a real data structure that is dynamically build and can be used in a lot of different ways.

type entrytype =
| Dir of entry list   (* directory with a list of files *)
| File of string      (* a file inside a directory *)

and

entry = {
	mutable e_name: string;   (* name of a file or directory *)
	e_type: entrytype;        (* what type is this together with type
                                     related information *)
	e_atime: float;           (* last access time *)
	e_size: int64;            (* size of the file or size of all the matching
                                     files in the directory *)
}

The 'and' statement is used to glue the two definitions together. They are created at the same time so that 'entrytype' can include 'entry' and vice-versa. 'entrytype' can consist one of 2 things: a directory with a list of entries or a file with its type. The directory entry has a mutable name. This is can be used later on to change a filename info the full path to that file.

As with ANSI C, the operators for Boolean algebra are '(&&)' and '(||)'.

Recursion

let rec dirwrite el depth sortfn =
List.iter (
  fun e ->
    match e.e_type with
    | Dir lst ->
       if e.e_size <> Int64.of_int 0 then begin
         print_string ((String.make (depth*2) ' ') ^ "Directory " ^
           e.e_name ^ " = (" ^ filesize e.e_size ^ ")\n"); 
         dirwrite lst (depth+1) sortfn
       end
    | File string -> 
       print_string ((String.make (depth*2) ' ') ^ e.e_name ^
         " (" ^ filesize e.e_size ^ ")\n")
  ) (List.sort sortfn el)
;;

Here is the recursive ('rec') function 'dirwrite' that traverses a given tree 'el' and writes the result to the standard output. The parameter 'depth' indicates the amount of spaces to write a tree like structure of filenames. The function sorts all the lists with the given function 'sortfn'. The new language structure here is 'fun <parm-1> ... <parm-n> -> <expr>'. This construction creates a function without a name. The parameters of this function like construction can be used like a template to match pairs.

This function suppresses directories that are 0 bytes in size to reduce clutter.

Variables vs. definitions

(* List of global variables *)

let min_size = ref (Int64.of_int 0) and    (* minimum size of a file in bytes *)
    last_access = ref max_float and        (* last access time in seconds since 1970 *)
    has_type = ref "" and                  (* type of file to show or empty to
                                              show all *)
    name_match = ref "" and                (* regular expression to match the filename
                                              with; empty is show all *)
    name_regexp = ref (Str.regexp "") and  (* pre-calculated regular expression *)
    no_symlinks = ref false                (* don't follow symbolic links to
                                              directories *)
;;

This is a list of variables that can be changed due to the 'ref <expr>' construction. Normally definitions are just a label to their contents. These definitions are pointers to the memory and can be read by '!<variable>' and written by '<variable> := <expr>'. The parameters given to the program can make changes to the way the files are read.

let rec dirread path =
let list = ref [] and
    size = ref (Int64.of_int 0) in
try
let dh = opendir path in
while true do
  let file = readdir dh in
  if file<>".." && file<>"." && file<>"CVS" && String.sub file 0 1 <> "." then
  let s=stat (path^"/"^file) in
  if s.st_kind = S_DIR &&
    (not !no_symlinks || (lstat (path^"/"^file)).st_kind <> S_LNK)
  then
    let dir = dirread (path^"/"^file) in
    list := 
    { e_name = file;
      e_type = Dir (fst dir);
      e_atime = s.st_atime;
      e_size = snd dir
    } :: !list;
    size := Int64.add !size (snd dir)
  else if 
    (!has_type = "" || gettype file = !has_type) &&
    s.st_size > !min_size && 
    s.st_atime < !last_access &&
    (!name_match = "" || Str.string_match !name_regexp file 0)
  then begin
    list := 
    { e_name = file;
      e_type = File (gettype file);
      e_atime = s.st_atime;
      e_size = s.st_size;
    } :: !list;
    size := Int64.add !size s.st_size
  end
done;
(!list, !size)
with 
| End_of_file -> (!list, !size)
| Unix_error (EACCES, err, parm) -> (!list, !size)
;;

The following functions are introduced in the function 'dirread':

Small is beautiful

let rec flat el path =
List.fold_right (
  fun e ls ->
    match e.e_type with
    | Dir lst -> flat lst (path ^ "/" ^ e.e_name) @ ls
    | File string ->
        e.e_name <- (path ^ "/" ^ e.e_name);
        e :: ls
  ) el []
;;

This neat little routine 'flat' hits the tree 'el' flat on the ground. It takes every file from every branch and creates a single list of all the encountered files. This is done with one of the most versatile standard routines inside ocaml: the 'List.fold_right' routine. This routine introduces a state machine (scarab) that crawls over a list and operates on every encountered element. It produces a new structure (droppings) as a result -- in this case, a flattened list.

The construction '<record-field> <- <expr>' changes the contents of a mutable record field. Without mutable fields, you can mutate records only by creating a new one with lots of fields inherited from the old one. This is a shortcut for that.

let name_order a b =
compare a.e_name b.e_name
;;

let type_order a b =
let typea = match a.e_type with Dir ls -> "dir" | File tp -> tp and
    typeb = match b.e_type with Dir ls -> "dir" | File tp -> tp in
if compare typea typeb = 0 then
  compare a.e_name b.e_name
else compare typea typeb
;;

let atime_order a b =
compare a.e_atime b.e_atime
;;

A set of sorting functions to use inside 'dirwrite'. The function 'compare' results in the widely used values of -1 for lower than, 0 for equal and +1 for higher than.

Command line parameters

let dir = ref "." and
    sort = ref name_order and
    show = ref 0
    in

Arg.parse [
   ("-t",Arg.Unit (fun () -> sort := type_order), 
     "Sort by type and filename");
   ("-l",Arg.Unit (fun () -> sort := atime_order),
     "Sort by last access time");
   ("-n",Arg.Unit (fun () -> show := 1),
     "List filenames");
   ("-b",Arg.Unit (fun () -> show := 2),
     "List both filenames and sizes");
   ("-s",Arg.Unit (fun () -> no_symlinks := true),
     "Don't follow symbolic links");
   ("--before",Arg.String (fun s -> last_access := converttime s),
     "Last access older than give date (format 'yyyy-mm-dd' or 'mmm yyyy')");
   ("--size",Arg.Int (fun i ->
        min_size := Int64.mul (Int64.of_int i) (Int64.of_int (1024*1024))
     ), "File size bigger than size in Mbytes");
   ("--type",Arg.String (fun s -> has_type := s),
     "File is specific type");
   ("--name",Arg.String (fun s ->
        name_match := s; name_regexp := Str.regexp (s ^ "$")
     ), "Filename matches regular expression")
] (fun d -> dir := d) "show [DIR]";
let res = dirread !dir in
if !show=0 then begin
  dirwrite (fst res) 0 !sort;
  print_string ("Total size " ^ filesize (snd res) ^ "\n")
end else
  List.iter
    (fun e -> 
      print_endline (e.e_name ^ if !show=2 then " ("^filesize e.e_size^")" else "")
    ) (List.sort !sort (flat (fst res) !dir))
;;

And here is the main routine. It calls the Arg.parse routine to parse the parameters given to the program. But this is too much un-GNU for me. I wrote my own version of it that follows the GNU coding standards a bit more than the default one (Gnuarg). The other version is a bit more complicated so I will include only the sources that use it.

Generating binaries

The code can be obtained from here. Just unpack it somewhere with 'tar -xzf show.tar.gz' and move into the source directory with 'cd show/src'. There is also a Makefile that compiles to machine code and installs everything. But Makefiles are too rough for sour eyes to show in this article. The nitty-gritty details are there in the source. The general compile form is.

ocamlopt -o show unix.cmxa str.cmxa basics.cmx show.ml

The only non-standard libraries in use here are unix.cmxa and str.cmxa.

make
su
make install
exit
show --help
show -s ~ --size 3 --before "apr 2003"

That concludes this example program.

Language features

Garbage collector

Just forget variables that contain complete data structures. Once it gets out of scope, the total structure will be eliminated from memory in due time.

Flexible data-structures

Any 2 data structures can be combined without hassle. Just create an array of records that contain 2 fields with hash tables of strings. No problem there... everything in a single variable than can be passed to functions or can be used globally in the program.

No pointers needed

A variable can have any type and when a new variable is created

Flexible in language boundary checks

The language can check array and string boundaries automatically, or those checks can be turned off for an extra speed boost. Without it, the language can give a segmentation fault, but that is the programmer's choice.

High quality error handling

Totally integrated into the language and no notable performance hit.

Native code generator and byte code interpreter

All the tools are there -- interpreter (ocaml), byte code (ocamlc) and native code compiler (ocamlopt) -- every wish is granted. The package comes also with a documentation generator (ocamldoc) and a simple to use profiler (ocamlprof) that adds usage counts as comments to the original source code. The language is also compatible with the more sophisticated profilers around.

ANSI-C compatibility layer

It is possible to include ANSI C routines inside OCaml programs, and OCaml routines inside C programs. This has a very easy to use API. Slightly less easy is the creation of OCaml data structures inside C; for me, that was the source of many segfaults. So, my routines call exported OCaml routines to fill data structures and create only OCaml strings and numbers in C. That way I won't have the hassle to debug the C code... OCaml is much easier to debug for me.

Object orientation

Not my favourite programming paradigm, but it is possible to build object-oriented programs in this language. Those features are not part of this article. I can live without them.

An active mailing list

This list is at caml-list@inria.fr and is normally in English. Yes, this originally French project has taken the burden to translate almost everything they got. This is no easy feat for them, so be grateful.

Cons:

Duplicate efforts in libraries

There are separate libraries for different type of big arrays, big files, and extra long integers. This isn't a big problem, because you can always just start with the normal structures and drop in the special library when need arises. The naming of the different functions is very much standardized, so renaming of function calls isn't needed much. The extra long integers though are too much different from normal integers. That part of the standard functions really need some tuning.

Readability

You need to be familiar with the basis constructions of the language, to make any sense of the actual code. Some constructions can look really weird without intimate knowledge of the language. OCaml is not a very natural language and has a very powerful, short notation for things. But this not much worse than languages like ANSI C, Perl, or lisp.

Not known enough in the Linux world

This language has excellent interfaces to standard libraries and easy binding to ANSI C, but still isn't very known. I like to create some articles like this to change that a bit. This is a really great language to program in, and gives you real power without the pitfalls common in other languages. Programmers should give it a try and feel that power once.

Developer at a small technology firm in the Netherlands called V&S bv. (www.v-s.nl) We sell firewall, anti-virus and spam boxes based on the Linux OS. Using more and more the OCaml language to write my applications. Busy writing a lightweight http server with an internal scripting language (camlserv.sourceforge.net, source code here) Interested in writing AI based computer games. Always trying writing one, nothing ready yet.