C CGI Library 1.2

Contents

Introduction

C CGI is a C language library for decoding, storing, and retrieving CGI data passed by the web server via the CGI interface. The library also has several handy data conversion functions.

Author: Stephen C. Losen, University of Virginia

C CGI Library Features

Query Strings and URL Encoding

A query string is a list of CGI variable names and values in application/x-www-form-urlencoded format, which looks like this: name1=value1&name2=value2&name3=value3 ... Each name and value is URL encoded as follows. Letters and digits are not changed. Each space is converted to +. Most other characters become %xx (percent followed by two hexadecimal digits) where xx is the numeric code of the character. For example, Help Me! is URL encoded as Help+Me%21. Note that in a query string the = and & characters between names and values are not URL encoded. However, if a name or value itself contains a = or &, then it is URL encoded using %xx.

When decoding query strings, the C CGI Library is tolerant of lax %xx encoding. It accepts most literal punctuation characters except for + and &, which must be encoded. It accepts literal = in variable values and it accepts literal % when not followed by two hexadecimal digits. For example, var=20%=1/5 is treated like var=20%25%3D1%2F5 and sets the value of var to 20%=1/5. A string that is not followed by = is a variable name whose value is "", a zero length string. For example, str1&str2& ... is the same as str1=&str2=& ..., where str1 and str2 are variable names whose value is "".

CGI Data Representation and Conversion

For simplicity and ease of use, most C CGI Library functions accept and/or return null terminated strings. You can easily convert a string to a numeric data type with the standard C library functions atoi(), atof(), strtol(), strtod(), etc. And you can convert a numeric data type to a string with sprintf().

Unfortunately, null terminated strings are not suitable for storing raw binary data, because a null byte in the data is mistaken for the string terminator. URL encoded strings containing %00 do not decode correctly because %00 results in a null byte. You can still manipulate binary data if you encode it beforehand, and the C CGI library has functions for encoding/decoding base64 and hexadecimal.

C CGI Library Data Types

In your C source you include the ccgi.h header file, which declares these data types.

CGI_varlist
is a list (lookup table) of CGI variables and/or cookies. Each list entry is a name and one or more values, where names and values are all null terminated strings. A name may have multiple values because 1) some HTML form elements, such as checkboxes and selections allow the user to choose multiple values and 2) the same name can be given to multiple form input elements. A CGI_varlist lists variable names and values in the same order that they are stored. In practice this ends up being the order of the input tags in the HTML form, but there is no requirement that browsers must preserve this ordering.
CGI_value
is a read only pointer to a read only null terminated string (const char * const). The CGI_lookup_all() function returns a null terminated array of these pointers.

C CGI Library Functions

The C CGI library provides these functions.

Except for CGI_prefork_server(), the C CGI library functions are reentrant because they do not modify any global variables or use any static local variables, so you can use these functions with threads.

Some functions accept null terminated string parameters of type const char *. These functions make copies of strings as necessary so that after the function returns you can safely do anything you want with any string that you have passed as a parameter. Some functions return null terminated strings of type const char * and you should not modify these strings.

CGI_varlist *CGI_get_query (CGI_varlist *varlist);
CGI_get_query() decodes CGI variables in the QUERY_STRING environment variable and adds them to variable list varlist. QUERY_STRING is presumed to be in application/x-www-form-urlencoded format. If varlist is null then a new variable list is created and returned, otherwise varlist is returned. Null is returned if varlist is null and QUERY_STRING does not exist or contains no CGI variables.
CGI_varlist *CGI_get_post(CGI_varlist *varlist, const char *template);
CGI_get_post() reads and decodes CGI variables from standard input and adds them to variable list varlist. If varlist is null then a new variable list is created and returned, otherwise varlist is returned. Null is returned if varlist is null and standard input is empty or contains no CGI variables. The template parameter (which may be null) is a file name template string that is passed to the standard C library function mkstemp() when uploading a file. (See the file upload section for more information.) CGI_get_post() checks the CONTENT_TYPE environment variable to get the data encoding, which is either application/x-www-form-urlencoded or multipart/form-data.
CGI_varlist *CGI_get_cookie(CGI_varlist *varlist);
CGI_get_cookie() parses HTTP cookies from the HTTP_COOKIE environment variable and adds them to variable list varlist. If varlist is null then a new variable list is created and returned, otherwise varlist is returned. Returns null if varlist is null and HTTP_COOKIE does not exist or contains no cookies.
CGI_varlist *CGI_get_all(const char *template);
CGI_get_all() calls CGI_get_cookie(), CGI_get_query() and CGI_get_post(), returning all of the CGI variables and cookies in one variable list. The template parameter (which may be null) is passed on to CGI_get_post().
CGI_varlist *CGI_decode_query(CGI_varlist *varlist, const char *query);
CGI_decode_query() decodes CGI variables in null terminated query string query (which is in application/x-www-urlencoded format) and adds the CGI variables to varlist. If varlist is null then a new variable list is created and returned, otherwise varlist is returned. Returns null if varlist is null and query is null or has no CGI variables.
CGI_varlist *CGI_add_var(CGI_varlist *varlist, const char *name, const char *value);
CGI_add_var() adds an entry named name with value value to variable list varlist. If varlist is null, then a new variable list is created and returned, otherwise varlist is returned. If the variable list already has an entry named name, then the value is added to that entry. This function is provided so that you can add data to a variable list by hand, or create a variable list for other purposes.
CGI_value *CGI_lookup_all(CGI_varlist *varlist, const char *name);
CGI_lookup_all() searches varlist for an entry whose name matches name case sensitively and returns all the values of the entry, or returns null if no entry is found. If name is null, then the values of the entry most recently visited by CGI_first_name() or CGI_next_name() are returned. The return value is a null terminated array of pointers to null terminated strings. The array and strings are stored in memory allocated to varlist, which you should not modify. The return type CGI_value * (const char * const *) declares the array and strings to be read only to discourage modification.
const char *CGI_lookup(CGI_varlist *varlist, const char *name);
CGI_lookup() searches varlist for an entry whose name matches name case sensitively and returns the first (or only) value of the entry, or returns null if no entry is found. If name is null, then the first value of the entry most recently visited by CGI_first_name() or CGI_next_name() is returned. If you expect an entry to have a single value, then this function is easier to use than CGI_lookup_all() and it is more efficient because it doesn't construct an array to return multiple values. You should not modify the string returned by this function.
const char *CGI_first_name(CGI_varlist *varlist);
CGI_first_name() begins an iteration of varlist and returns the name of the first entry, or returns null if varlist is null. You can get all the values of this entry with CGI_lookup_all(varlist, 0); You should not modify the string returned by this function.
const char *CGI_next_name(CGI_varlist *varlist);
CGI_next_name() continues an iteration of varlist and returns the name of the next entry. You can get all the values of this entry with CGI_lookup_all(varlist, 0); Returns null if 1) there are no more entries, or 2) varlist is null, or 3) no iteration was started with CGI_first_name(), or 4) new data was added to varlist during the iteration. You should not modify the string returned by this function.
void CGI_free_varlist(CGI_varlist *varlist);
CGI_free_varlist() frees all memory used by variable list varlist.
char *CGI_encode_query(const char *keep, const char *name1, const char *value1, ..., (char *)0);
CGI_encode_query() returns a query string in application/x-www-form-urlencoded format that is built from a null terminated list of null terminated string arguments. The first argument keep (which may be null) is a null terminated string that specifies characters that you do not want to URL encode with %xx. You do not need to specify letters or digits because they are never encoded. The first two arguments after keep are a name and value pair, the next two are a second name and value pair, etc. Be sure to terminate the argument list with (char *)0. (When passing a variable length argument list, the C compiler does not automatically cast 0 to (char *)0, which is necessary on 64 bit platforms.) In the result the names and values are URL encoded and separated with literal & and = characters like this: name1=value1&name2=value2 ... Memory is allocated with malloc() to hold the result, which you should free with free().
char *CGI_encode_varlist(CGI_varlist *varlist, const char *keep);
CGI_encode_varlist() returns a query string in application/x-www-form-urlencoded format that is built from CGI_varlist varlist. The argument keep (which may be null) is a null terminated string that specifies characters that you do not want to URL encode with %xx. You do not need to specify letters or digits because they are never encoded. The names and values in varlist are URL encoded and separated with literal & and = characters like this: name1=value1&name2=value2 ... Memory is allocated with malloc() to hold the result, which you should free with free(). You can use CGI_add_var() to build varlist.
char *CGI_encrypt(const void *p, int len, const char *password);
CGI_encrypt() encrypts input p of length len bytes using password to generate the cipher key. Also computes a message digest using the input data. Returns the encrypted digest and encrypted data in a base64 encoded string, which must be decrypted with CGI_decrypt() and the same password. Returns null if p is null or if len is less than one or if password is null or zero length. Memory is allocated with malloc() to hold the result, which you should free with free(). (See the cryptography section for more information.)
void *CGI_decrypt(const char *p, int *len, const char *password);
CGI_decrypt() decrypts input p, which was encrypted with CGI_encrypt(), using password to generate the cipher key. The output is a message digest and decrypted data bytes. Verifies the data using the message digest and returns the data. Returns the length of the data in *len. Returns null if p cannot be decrypted and verified. Also returns null if p or password is null or zero length. Memory is allocated with malloc() to hold the result, which you should free with free(). (See the cryptography section for more information.)
char *CGI_decode_url(const char *p);
CGI_decode_url() returns a URL decoded copy of input string p. Memory is allocated with malloc() to hold the result, which you should free with free().
char *CGI_encode_url(const char *p, const char *keep);
CGI_encode_url() returns a URL encoded copy of input string p. Memory is allocated with malloc() to hold the result, which you should free with free(). The keep argument (which may be null) is a null terminated string that specifies characters that you do not want to URL encode with %xx. You do not need to specify letters or digits because they are never encoded.
char *CGI_encode_entity(const char *p);
CGI_encode_entity() returns a HTTP entity encoded copy of input string p. Memory is allocated with malloc() to hold the result, which you should free with free(). CGI_encode_entity() makes the following conversions: < becomes &lt;, > becomes &gt;, & becomes &amp;, " becomes &quot;, ' becomes &#39;, newline becomes &#10;, and return becomes &#13;.
char *CGI_encode_base64(const void *p, int len);
CGI_encode_base64() encodes input p of length len bytes, and returns the result, which is a null terminated base64 encoded string. Memory is allocated for the result with malloc(), which you should free with free(). Base64 is a commonly used encoding that represents arbitrary bytes of data using the following printable characters: upper case, lower case, digits, +, /, and =.
void *CGI_decode_base64(const char *p, int *len);
CGI_decode_base64() decodes p, which is a null terminated base64 encoded string, and returns the result. The length of the result is stored in *len and a null byte is written just after the last byte of the result. Memory is allocated with malloc() to hold the result, which you should free with free().
char *CGI_encode_hex(const void *p, int len);
CGI_encode_hex() encodes input p, of length len bytes, and returns the result, which is a null terminated hexadecimal encoded string. Memory is allocated for the result with malloc(), which you should free with free(). Hexadecimal is a commonly used encoding that represents arbitrary bytes of data using two hexadecimal digits for each byte.
void *CGI_decode_hex(const char *p, int *len);
CGI_decode_hex() decodes p, which is a null terminated hexadecimal encoded string, and returns the result. The length of the result is stored in *len and a null byte is written just after the last byte of the result. Memory is allocated with malloc() to hold the result, which you should free with free(). Returns null if p is null, or if the length of p is odd, or if p contains characters other than hexadecimal digits.
void CGI_prefork_server(const char *host, int port, const char *pidfile, int maxproc, int minidle, int maxidle, int maxreq, void (*callback)(void));
CGI_prefork_server() implements a SCGI (Simple CGI) pre forking server. The host specifies a local network address, either by hostname or dotted decimal IP address, and port specifies a TCP port number. The SCGI server listens for requests on the specified address and port. If host is null, then the server listens on all local addresses. The pidfile (which may be null) is an optional file name where the server writes its process ID. The SCGI server forks up to maxproc child processes to handle requests. It forks and destroys processes to maintain between minidle and maxidle idle processes. Each process exits after handling maxreq requests. If maxreq is less than one, then it is unlimited. You provide the callback function, which the SCGI server calls to process each web request. CGI_prefork_server() does not return unless it fails. (See the SCGI server section for more information.)

Using the C CGI Library

Here is an example program that outputs all of its CGI data. In your C source, include ccgi.h and link your program with libccgi.a. (If you use CGI_encrypt() or CGI_decrypt() then you must also link with the openssl library libcrypto.) The simplest way to obtain your CGI data is with CGI_get_all(). If you are not uploading any files, then just pass it a null argument.

#include <stdio.h>
#include <ccgi.h>

int main(int argc, char **argv) {
    CGI_varlist *varlist;
    const char *name;
    CGI_value  *value;
    int i;

    fputs("Content-type: text/plain\r\n\r\n", stdout);
    if ((varlist = CGI_get_all(0)) == 0) {
        printf("No CGI data received\r\n");
        return 0;
    }

    /* output all values of all variables and cookies */

    for (name = CGI_first_name(varlist); name != 0;
        name = CGI_next_name(varlist))
    {
        value = CGI_lookup_all(varlist, 0);

        /* CGI_lookup_all(varlist, name) could also be used */

        for (i = 0; value[i] != 0; i++) {
            printf("%s [%d] = %s\r\n", name, i, value[i]);
        }
    }
    CGI_free_varlist(varlist);  /* free variable list */
    return 0;
}

File Uploads

To upload files to your CGI program, your HTML form must use the post method and must specify multipart/form-data encoding, so the form tag looks like this:

<form method="POST" enctype="multipart/form-data"
    action="url-for-your-CGI">

Within the HTML form a file upload tag looks like this:

<input type="file" name="uploadfield" />

Most browsers render this tag with a file browse button and a text field to enter and/or display the name of the file being uploaded.

When the user submits the form, the browser sends the file data together with any CGI form variables using multipart/form-data encoding. To receive uploaded file data you must call CGI_get_post() or CGI_get_all(), and pass a file name template string, a copy of which is passed on to standard C function mkstemp(). The final six characters of the template string must be XXXXXX and mkstemp() replaces these with random characters to create a new file with a unique name. If you pass a null or invalid template string, then uploaded file data is silently discarded.

CGI_get_post() or CGI_get_all() stores two names for the uploaded file in the variable list, which you can retrieve with

value = CGI_lookup_all(varlist, "uploadfield");

This returns an array of two strings (provided no other form input tags are named uploadfield). In value[0] is the name of the uploaded file on the web server, which is derived from the template string. In value[1] is the name of the file specified by the user in the browser. If the user has not uploaded a file, then varlist has no entry named uploadfield and CGI_lookup_all() returns null.

We use mkstemp() to guarantee unique file names because a form may have multiple file upload fields, resulting in multiple files. Furthermore, multiple users can upload files to multiple instances of the CGI at the same time. Here is an example CGI program that uploads a file.

#include <ccgi.h>
#include <stdio.h>

int main(int argc, char **argv) {
    CGI_varlist *varlist;
    CGI_value *value;

    fputs("Content-type: text/plain\r\n\r\n", stdout);
    varlist = CGI_get_all("/tmp/cgi-upload-XXXXXX");
    value = CGI_lookup_all(varlist, "uploadfield");
    if (value == 0 || value[1] == 0) {
        fputs("No file was uploaded\r\n", stdout);
    }
    else {
        printf("Your file \"%s\" was uploaded to my file \"%s\"\r\n",
            value[1], value[0]);

        /* Do something with the file here */

        unlink(value[0]);
    }
    CGI_free_varlist(varlist);
    return 0;
}

Simple Cryptography Support

HTML and HTTP provide no native support for protecting and verifying CGI data. Many web applications pass state data to the browser in cookies or form variables. When the browser passes this data back, the web application cannot tell if it has been tampered with. An attacker can easily handcraft a web request that includes forged cookies or forged form data.

The C CGI Library addresses this problem with CGI_encrypt() and CGI_decrypt(). CGI_encrypt() computes a SHA1 message digest from the input data, encrypts the digest and the input data, and returns the result in a base64 encoded string. (Raw encrypted output is binary.) CGI_decrypt() reverses the process. It decrypts the digest and the data, and recomputes the digest. If the two digests match, then it returns the data. Otherwise it returns null to indicate failure. CGI_encrypt() and CGI_decrypt() use a password that you provide, which is a null terminated string of arbitrary length (the longer the better). It is essentially impossible to tamper with the data without knowing the password. If the output of CGI_encrypt() is modified in any way, then CGI_decrypt() computes a message digest that does not match and returns null.

To protect state data, simply encrypt it with CGI_encrypt() and a password before passing it to the browser. When the browser passes the encrypted data back, decrypt with CGI_decrypt() and the same password. If CGI_decrypt() succeeds, then you know that the data has not changed. Of course the security of the data depends on the security of the password, which should be very difficult to guess and very difficult to steal.

CGI_encrypt() uses the openssl library libcrypto and has these features:

Pre Forking SCGI Server

Usually when a web server receives a request for a CGI resource, the web server executes a CGI program, which handles the request and exits. This does not perform well under high load. SCGI (Simple CGI) is a protocol for running a persistent CGI server. When a web server receives a request for a SCGI resource, the web server connects to a SCGI Server and forwards the request using the SCGI protocol. The SCGI server responds to the web server, which forwards the response back to the user's browser. This is much more efficient than executing a CGI program for each request. To configure the Apache httpd web server to use SCGI, see the mod_scgi module.

Under high load a SCGI server must be able to handle multiple requests concurrently. The SCGI server provided here pre forks a specified number of child processes that all wait for requests. The parent process monitors how many child processes are busy and creates more if necessary. If too many processes are idle, then the parent terminates some of them.

You provide the code that handles web requests in a callback function, which is called once for each request. The environment, standard input, and standard output are set up so that your callback can be written very much like a traditional CGI program. All the functions in the C CGI library work as specified when using the SCGI server.

To start the SCGI server, call CGI_prefork_server() and pass it a pointer to your callback function. The SCGI server puts itself into the background, and forks child processes to handle web requests. If CGI_prefork_server() returns, then it has failed. The web server does not automatically start the SCGI server program, so you must start it. You control which user account runs the SCGI server and what privileges it has.

To terminate the SCGI server, send the SIGTERM signal to the parent process and it sends the signal on to its child processes and exits. The parent process writes its process ID in a file if you pass the file name to CGI_prefork_server() in pidfile.

The callback function operates very much like a traditional CGI program, except that it gets called multiple times. When writing your callback consider the following.

Here is an example SCGI server.

#include <ccgi.h>
#include <stdio.h>
#include <syslog.h>

static void
cgi_callback() {
    static int first_call = 1;
    CGI_varlist *varlist;

    if (first_call) {
        first_call = 0;

        /* initializations for each child process go here */

    }
    varlist = CGI_get_all(0);

    fputs("Content-type: text/html\r\n\r\n", stdout);

    /* write the rest of the web response to stdout */

    /* free memory and close open files */

    CGI_free_varlist(varlist);
}

int
main(int argc, char **argv) {

    /* initializations before forking child processes go here */

    openlog("my-scgi-server", 0, LOG_DAEMON);

    CGI_prefork_server("localhost", 4000, "/var/run/my-scgi-server.pid",
        /* maxproc */ 100,  /* minidle */ 8,  /* maxidle */ 16,
        /* maxreq */ 1000, cgi_callback);

    /* if CGI_prefork_server() returns, then it failed */

    return 0;
}