I think that this writeup will stick to C/C++ code, since most of the code I write that I actually make an effort to make portable is one those languages. If someone has experience writing portable Perl, or Bourne Shell, or Python, or whatever, feel free to add a WU in here.
I'm skipping over a lot of stuff, particularly anything GUI related, because I don't do GUIs. This is mostly low-level stuff, and not very complete at that.
These rules may seem obvious to you. They usually seem pretty obvious to me. But I've had to deal with enough code that didn't follow them (usually cleaning up the subsequent mess), that I thought this node might be useful.
Rule #1: Don't do stupid things
There are lots of things in C and C++ that are implementation defined. That means it varies depending on what compiler you're using, and probably what CPU it's going to be running on. Among these are the sizes of basic types, and the endianess of the system.
There are fairly easy workarounds for these. First, use typedef so that when the size of a basic type changes (and it will, somewhere, somehow), it's an easy one line fix. I've seen 50,000+ lines of C++, much of which assumes long is 32-bits. That code would require an insane amount of work to port to something like a DEC Alpha (or Clawhammer, for that matter). The switch from Windows 3.1 to Windows 95 was a lot harder than it should have been because of this problem (among other things).
You can save yourself from endian issues pretty easily. For example, don't take the address of a long, assign it to a char*, and then read the bytes out through the char*. Instead extract them with bit shift operations and the AND and OR bitwise operations; this is portable, and not much slower. Why is the first version (very, very) bad? Let's say the long contains the hexadecimal value 0x11223344, and the char pointer is named "c". On a little endian system (like x86), c will be 0x44, but on a big endian system, c will be 0x11. Hopefully you can see why this might be a problem.
Similarly, never cast something like a char* to a long*; not only does this raise endian issues, but on some machines, like SPARC, it can cause a trap to be executed, totally trashing performance, or possibly causing your process to die on the spot. In general, be wary of pointer conversions, especially between basic types. Casting things to or from void* is OK (sometimes), but pretty much anything else is just asking for it.
Avoid screwy things like bitfields, unless you need compatibility with hardware or something. Bitfields are a real mine field of problems.
Basically, stick to some sort of reasonable API. If it's mostly pure ISO C or C++, but with some POSIX code here and there, try to make the POSIX stuff in it's own little space, where you can swap it out when you try to port it to VMS.
If you can, avoid really OS dependent stuff, like Unix calls not standardized by POSIX or Single Unix, or those really obscure Win32 calls. This is basically impossible in most real programs, so just try to keep it contained (and well documented).
Don't code things you don't know how to do (this can be restated as "Know your limits -- you're not as good as you think you are"). Seems obvious, I know. But people (myself included) often fall into the trap of re-implementing things that are right there, just begging for us to reuse them. For example, there is a certain application, which I have happen a passing familiarity with, that uses OpenSSL for doing authentication with RSA keys, and also SSL encryption between the client and the server. Another part of the application needed to use the SHA-1 hash function. You might think that the person who wrote this section of the code would have simply called out to the library, since we were already requiring it be available. Instead, he wrote his own implementation. Which a) was about 1/50 the speed of the OpenSSL implementation, and b) was wrong.
One nasty one that I think I'll share with you is an odd thing in the select call on Linux systems. On most Unix machines, the timeout variable is a constant (and POSIX.1g defines it that way), but on Linux the value of the timer upon return reflects the amount of time that was spent waiting. For best portability, you should assume the timer value is undefined upon return, and reinitalize the value. I recently ran into code at work that had this problem. The timer value would eventually hit zero (the select call had waited until timeout, in our case 1/4 of a second), and then the server would start going into a constant loop without blocking at all, taking up all the CPU we could give it.
This is the really sure way; you won't ever get things really "right" until you actually sit down and run the code on some bizarre system.
Basically try as many different combinations of compilers, operating systems, and CPUs as you can get your grubby little mitts on. Trust me, you'll find your portability problems pretty fast. And if you paid a bit of attention while you were writing your code, hopefully they won't prove too fatal.
The Practice of Programming, by Brian W. Kernighan and Rob Pike, has some excellent suggestions on portability, testing, design, and debugging. I stole some of the ideas in this node from there, others I have learned the hard way.
A Funny Story
I once had the great displeasure of using a really horrible application called PC^2. Believe me, pretty much any piece of software you've ever used, no matter how bad, is better than this. But I digress... This particular POS was written in Java. Nice and portable, right? Write once, run anywhere, and all that jazz. Our site had the "honor" of being the first Unix setup to run it. Soon after it was handed off the users (we were on very tight deadlines, mind you), we started seeing failures -- PC^2 wanted to give people an editor to use, so it was exec'ing C:\windows\notepad.exe.
Horay for portability, right?