Literally »crude« or »rough«; assumed to be the truncation of gladius rudis, that is, a crude sword. The rudis was the wooden stick used by gladiators to practice fencing; it is commonly depicted in post-Roman art as a carefully replicated gladius, but is more likely to have been just a short round stick with some sort of dish or bar for a guard. (Lewis & Short, in fact, explicate the word only as »A staff used by soldiers and gladiators in their exercises (perh. a wooden sword)«.)
Besides its practical use, the rudis also had a ceremonial function as the gift conferred on a gladiator who by repeated victories had won his freedom; a parallel to the freedman's cap. The function of tokens of this species might (I can never tell) require some elaboration: now I think it seems to many people to be given as a lagniappe along with the real and valuable prize of liberty, but in those days written contracts were very rare, so that the sword (or cap) formed the physical embodiment and container of the freedom, a concrete object to contain the abstract the way a jar contains pickles. The thing therefore was apparently considered not only equally precious as, but in some sense identical to the conferred right. I dwell on this because although it is something which I find difficult to explain with precision, it is an important distinction between the past and modern world; this idea suffused society and is now quite extinct. I originally set out here on a long divagation, but the discussion belongs elsewhere than a node on wooden swords; in idolatry, perhaps.
Returning therefore with haste to the rudis, such a gladiator as had received it was called a rudiarius; some continued to fight. Others became lanista, gladiator-owners; others again took up positions as fencing-masters in another lanista's establishment (these were also called rudis, in fact; summa rudis was the foremost, secunda rudis his auxiliary, and presumably so on, although a perfunctory flip through the books yields no evidence of a third man), or arena functionaries. Those that did stay in the ring typically became very popular; it's good to see a skilful man fight.
In modern times I understand that it's a sometime tradition in certain units of military men to give the man who has honorably served out his time a rudis; this sword is of course of the type of the fancy, wooden replica of a gladius, and is (appears to me to be) given as just that decorative memento people mistake the original rudis for. Nevertheless, considering the history of the custom there is a pleasing strain of mordant humor in it, the sort which is quite common among those who have had to deal with worse in this world than an unkind word.