(thing) by arcterex Tue Oct 30 2001 at 22:28:24
strtok is a C/C++ function that allows a programmer to parse parts of a string on user specified characters (tokens). It is part of the string library string.h. strtok() has the following format:

char *strtok(char *s, const char *delim);

The string 's' is (on the first call only) the original string that will be parsed. String 'delim' is a string of characters that will be used as tokens.

Simple Example 

char* util = 0;
util = strtok( "this is a test", "s\n" );
printf("Token = '%s'\n", util );

This code results in:
Token = 'thi'

This is a very simple example. One of the biggest things about strtok() that has screwed this noder up is that subsequant calls to strtok() must have the first argument set to NULL. When no more tokens can be found in the string, NULL is returned.

Slightly Less Simple Example 

char* util = 0;
char buffer = "This is a test\n";
char tokens = "s\n";

/* First call has the first argument of the string to examine */
util = strtok( buffer, tokens );
do 
{
   printf( "Token = '%s'\n", util );
}
while( util = strtok( NULL, tokens ));

This code results in:
Token = 'Thi'
Token = ' i'
Token = ' a te'
Token = 't'

Each occurance of characters in the string 'tokens', in this case 's' and '\n', is removed from the string 'buffer' and the characters between them is returned as a character pointer. Note that the first call is made with 'buffer' as the first argument, but later calls (in the do{}while() loop) are made with NULL as the first argument. If this wasn't done, the first token would always be returned!

The only time when this function will have problems is if you are doing parsing within parsing. If you parse out a token from a string, and then try to use strtok() on that token (while still in the loop through the original string), Bad Things happen. This is because strtok() uses a static buffer for it's parsing. The same applies to multi-threaded applications using this function. If you are going to have the possibility of this happening, use strtok_r(), which is thread safe, instead.

I like it!