As a non-non-programmer in one of these two languages, there are a few points I feel must be added to the commendable effort of SPasmofiT to explain this sticky matter. This writeup is intended as a collection of footnotes to the above. Relevant parts will be quoted. Read disclaimer below!
These programming languages are meant to satisfy the same kind of needs and are best suited to a certain type of project, the "enterprise application".
This is not completely true. The languages have more applications than "enterprise programming" and Java was by no means intended to have such a limited field of application. It is possible to create things like 3D rendering engines in Java for instance. Fairly good ones at that. It is possible to create server applications, text editors and compilers. The standard Java libraries even have classes for building compilers, which proves the point that the designers had much more in mind than enterpise applications. Even complete software development environments exist that were written in Java. Java was developed to be a normal, full blown third generation programming language. It differed from existing languages in two pivotal points: Java executable files run on any platform (that supports a Java Virtual Machine) and the syntax is more elegant than most other OO languages.
Both Java and C# are 4th generation languages.
Java and C# are not fourth generation languages. Fourth generation languages, like PowerBuilder for example, or perhaps Rational Rose RT, offer fully programmed building blocks which can be glued together by a programmer to form rounded applications. Usually, a programmer doesn't even need to know any special language to do so. In Rose RT for instance, the programmer draws a diagram of an application on the screen and Rose RT creates the actual executable from your drawing. Java and C# are much more low level than this. Let's illustrate this by saying that you have operators for shifting individual bits left and right in a register. This is about as low level as one can get. Java and C# do have extensive libraries that encapsulate much of the tedious jobs that come with low level programming, like drawing buttons on a screen, playing sounds, painting pixels on a screen, etc. Java and C# are more like third generation languages which come with a lot of standard tools.
Java and C# by themselves are more or less useless.
Because of the high degree of abstractisarion used, they both relay on a set of applications and libraries that connect the language to the OS (operating system) and different data sources (databases, directories, etc.).
This is both true and not true. All languages, including C and even assembly, require help from the operating system to perform anything at all. Sometimes this help comes in the form of direct system calls, but most of the time additional platform libraries are used. Traditional C cannot even output text to the screen without a system dependent library. Further, the higher level types of libraries that connect Java to database systems also exist for C or other languages. A C program needs these to communicate with a database, just like Java does.
About 2 years ago (1999) the Microsoft Corporation (http://www.microsoft.com/) was offering several 3rd generation languages, including MS Visual Basic and MS Visual C++, bundled in the "MS Visual Studio Platform". These offered little regarding web technologies or portability, but were widely available because of their ease of use and powerful integration with MS Windows.
If one insists on calling C# and Java fourth generation languages, which we really shouldn't, then Visual Basic (especially VBA) was one too. That aside. Visual C++ had lots of web support and standard libraries. Microsoft even demonstrated that one could build a browser in under five minutes using Visual C++.(Note by the way that the language is called C++ and that Visual C++ is just the name of Microsoft's integraded development environment, a program for writing C++ programs). What became painfully clear from the Visual C++/MFC experience is that things like application wizards (which write 80% of your program for you) make it virtually impossible to write anything else but web browsers. The Java aproach offers much more flexibility and the methods used to encapsulate user interface design were much more stable, easy to use and less restrictive.
Interestingly enough, C# adds nothing to the portability of applications. C# applications will never run on anything but Windows platforms. It is however more elegant and clean than C++ and the standard libraries that came with Visual C++. It 'feels' more like Java in that respect. A platform dependent Java. What is even more remarkable is the fact that C# is even less platform independent than any of the now common programming languages, save assembly language. A C program can be compiled on any platform that supports a C compiler. So, while only executables that employ a virtual machine (like Java) work on several platforms, one single C code can be used to build unique executables for each platform, making C code source portable, where C# is not portable at all. Only minor adjustments are necessary at points where the program interacts with the operating system. With the POSIX standard, that has become even less cumbersome. By the looks of things, there will never be a C# compiler for anything but Windows. This means that people who want to develop for multiple platforms have to build one version of their application in C# and one in some other language. This is quite a shameless attempt to chain even more people to the Windows platform.
As far as performance is concerned, both Java and C# are too slow to be used in games or most "regular" applications, they are designed to be used for "n-tier systems".
This is true for gaming in Java, but it shouldn't be true for C# at all. Java has a program running 'between' your computer and the Java application. This is necessary in order to be able to run your Java program on a Macintosh, a Linux box and a Windows machine alike. This abstraction layer takes some of the computing resources so Java programs will always be a little slower than applications written in C. For anything but gaming, this is no problem however. Even for games, it will only be too slow for the most cutting edge technologies, but this naturally makes it an unatractive choice for the game developer. C# only runs on Windows so it doesn't need a Virtual Machine like this.
C# has some added programming features like pointers and operator overloading, that have been removed from Java because they were considered as problem-makers and obsolete, even in 1995.
This is an interesting point indeed. Opinions differ on this one. Being both a C++ and a Java programmer I must say that the nonexistence of pointers is very desirable at this level. Pointers are only useful in C++ for doing hardware dependent things, like rendering signals in close to realtime. This is not something that you'd want to do in Java or C#. Thus, pointers are not a 'feature' but a nuisance when writing something like a web application.
The recent success of open-source software (properly called "free software")
I can't help it: open source is not free software. There are people that would hang you for saying that!
Disclaimer: SPasmofiT makes several valid points and provides an interesting insight in this matter. I felt these comments were just a necessary addition to a good writeup. I did not intend to be pedantic.
A short response to sql*kitten's writeup below:
A short migration path from Java to C# isn't going to do much for the language I'm afraid. Software engineers don't go around looking for short migration paths all day. They preferably engineer software all day. They only migrate when there is something to gain from the procedure. Something that outweighs the cost of migration considerably. Unless C# provides some significant improvement over Java, no Java developers will make the switch. I for one am curious as to what major improvement C# brings over Java to give it a raison d'etre in the first place. It seems more likely that numerous Visual Basic programmers will make the switch, and so will C++ programmers that target Windows exclusively. Apart from those...?