As has previously been stated, the homogenous nature of Japanese culture often causes exclusion and prejudice against minority groups. This prejudice is not limited to Asian minority groups and frequently extends to non-Asian groups as well.
This phenomena can be relatively rare in such urban areas as Tokyo and Osaka or especially in tourist areas such as Kyoto and Nara. However, it is more than likely to happen to gaijin travelling and living in more rural areas or areas which tend to draw few tourists.
From my personal experience, if you are a white male in America and have never experienced the type of latent bigotry which so many African-American's encounter on a tragically regular basis, a brief time staying in Japan may be immeasurably helpful in bringing about a change in perception and understanding.
It may be hard to understand what I am talking about here without personal experience: The first time an old woman crosses the street just to avoid walking on the same side as you...well, needless to say, it can be quite a cold realization. When people cut in front of you in line while the clerk happily serves them first, it will help cement it home. When taxis refuse to take you anywhere because of the color of your skin, you will finally realize that you aren’t in Kansas anymore Toto.
Needless to say, this type of insular attitude is slowly disappearing as Japan becomes a more internationally minded country. But should you be considering locating yourself in a rural or non-urban setting inside Japan...well, you have been warned.