In the Torah, most Jewish Festivals have a set down length. Pesach (Passover) and Succot (Tabernacles) are seven days,and Shemini Atzeret (the day after Succot) and Shavuot (the "Feast of Weeks") are one day.
However, outside of Israel, Orthodox Jews add an extra day onto these. Succot is still celebrated for 7 days, because Shemini Atzeret is tacked on the end and this is celebrated for two days. Pesach is celebrated for 8 days, and Shavuot for two days. Additionally, Succot and Pesach have the main festival days, and also "intermediate days" - "Chol Hamoed" - where the laws are less strict. So in total we have festivals looking like this.
- Succot
- In Israel - One festive (Yom Tov) day, six intermediate days, followed immediately by one festive (Yom Tov) day of Shemini Atzeret.
- Rest of the world - Two festive (Yom Tov) days, five intermediate days, followed immediately by two festive (Yom Tov) days of Shemini Atzeret - although the second day is called Simchat Torah.
Pesach
- In Israel - One festive (Yom Tov) day, five intermediate days, one festive day.
- Rest of the world - Two festive (Yom Tov) days, four intermediate days, two festive days.
Shavuot
- In Israel - One festive (Yom Tov) day.
- Rest of the world - Two festive (Yom Tov) days.
Rosh Hashanah
- In Israel - Two festive (Yom Tov) days.
- Rest of the world - Two festive (Yom Tov) days.
Yom Kippur
- In Israel - One festive (Yom Tov) day.
- Rest of the world - One festive (Yom Tov) day.
To understand this, we have to understand why the differences.
In the times of the Temple in Jerusalem, about 2000 years ago, there was no set calendar for the Jews. A lunar calendar is used for the dates of the festivals, and the system relied on someone seeing the new moon, going to the Sanhedrin (the Jewish court of Sages) and telling them about it. They would ask him questions, and if they decided he was telling the truth, they would declare the new month had started. Then, everybody would know the dates of the festivals.
Once Jews moved outside of Israel, messengers had to be despatched on horseback to other places to pass on the message. Sometimes, they would take a couple of weeks to make the journey. This meant that these people wouldn't know what date to start a festival that began on, say, the 10th of the month. So, these people would start the festival based on the previous month's start and hope they were right. To be safe, though, it was decreed that they would keep an extra day of the festival. And this is where the second day of Yom Tov comes along.
This explains Pesach, Shavuot and Succot. Rosh Hashanah is two days even in Israel as it falls on the first day of the month (and year). So even in Israel they would have to guess based on the last month, and keep two days. Yom Kippur should be two days outside of Israel. But as it involves fasting for 25 hours, this was deemed unsafe and hence it is only kept as one day everywhere.
The final question is, now that 1) we do have a set calendar and 2) even if we didn't, we have telephones, why do we still keep two days of Yom Tov? I'll answer that with a joke.
How does a Rabbi change a lightbulb?
CHANGE????????
This whole issue brings up a big question of what to do if you're an Israeli spending a festival outside of Israel, or vice versa. The normal answer is that you keep what you do at home - an Israeli in England keeps 1 day, and an English person in Israel keeps 2 days. That said, a lot of otherwise religious people from outside Israel keep 1 day when they are there.
I believe most non-orthodox groups officially keep only 1 day of each festival as a Yom Tov day, on the basis that this is what the Torah says. Never-the-less, the views of the Rabbis are very strongly respected in Orthodox Judaism and hence Orthodox Jews to this day keep the extra day of Yom Tov.
To add a few comments to stagmeister's comments below.
If you are in Eretz Yisrael (Israel) for the whole year (that is, for the three pilgrimage holidays - Pesach, Sukkot, and Shavuot), then you are considered as "living in Israel." The same goes for if you live in Chutz La'artetz (outside of Israel) for all three pilgrimage festivals. So let's consider possible situations:
* You are an American, studying in Israel from September until June. However, Shavuot falls on June 15th, after you are already back in America. So, you practice two days of Yom Tov.
The most prevalent view I have heard is that if you are a non-Israeli spending a whole year in Israel (say for example learning in Yeshiva for a year after high school), then you still keep two days as you aren't living there. That said, there are differing opinions on this.
* You are an Israeli, who is going to America for Pesach to visit your cousins. So since you are only going to be there for one of the pilgrimage holidays, you are only obligated to have one seder and one day of Yom Tov on each end of the holiday.
I'd agree with this 100%. However, an Israeli in England or America, only keeping one day of Yom Tov, shouldn't publicly not keep Yom Tov on the second day. If he was religious, and wore a kippah, and was seen, say, driving around, this would be very confusing and sets a bad example to people who don't know he's Israeli. Therefore, it would be ok for him to sit upstairs in the home he's staying watching TV, but shouldn't do anything in public.
* You are an Israeli who is now working in America for your kibbutz, running a shrink-wrap factory in North Carolina. You're staying here permanently for a few years until it's set up properly, you can return to your kibbutz, and someone else takes your place. You will practice two days of Yom Tov.
Agreed.