An acronym for units in the last place. Floating point error depends crucially on the exponent. A floating point error of 1 ulp corresponds to the minimal expressible difference in the mantissa with the given exponent.
Say we use a base of 10 and a mantissa of precision 3. If we express Avogadro's constant as 6.03×1023 instead of the "true" value of 6.022×1023, we've made an error of 0.8 ulps. If we'd expressed it as 6.02×1023 we'd have an error of 0.2 ulps; this is in fact the best we can do with these parameters.
ulp is subtly different from relative error. But each can be bounded in terms of the other.