It is a popular urban myth, especially amongst the Christian fundamentalists who do not believe in evolution, that the appendix (the thing attached to your caecum, not the stuff at the end of books) is not a vestigial organ.
The appendix does not serve any endocrine function in the foetus. While the appendix does contain lymphoid tissue, it does so in amounts that are not significantly greater amount than other parts of the gastrointestinal tract. Children who have had appendices removed have not been shown to have any significant worsening of the maturation of their immune system.
Having an appendectomy done can lead to other long term sequelae, including the development of adhesions causing bowel obstruction in later life, but that has nothing whatsoever to do with the nature of the appendix itself but rather to the type of the operation; adhesions are a complication of all intra-peritoneal surgery where the bowel is handled.
Sources: my medical textbooks and conversations with surgeons.
Anatomically, the appendix is a blind end that is attached to your colon at your caecum. In some people, this tissue can get terribly inflamed causing appendicitis.