No, it's not impossible, it just needs a little bit of practice. A unicycle is not something you can have a quick go on and pick up in half an hour, unless you already have a well-honed sense of balance. It will be frustrating to start with; take your time. Persevere.
The first challenge to overcome is getting on the thing in the first place.
First, decide which foot is your favourite foot. Most people have a preference; a good way to find out is to put a foot onto a chair. Which foot did you use?
Make sure the seat is at the right height. A common mistake is to have it too low; this makes it harder, and makes your knees ache. The ideal height is so when you are sat on the seat you should be able to put your foot on the pedal at the bottom with your leg very nearly straight, but not locked.
You'll need something to lean on. Ideally you want something on both sides of you; between the backs of two chairs is good. Failing that, the long edge of a tabletop, or a wall.
Next, rotate the wheel so the pedal your favourite foot will go on is right at the bottom. Putting all your weight on the pedal at the bottom will stop the unicycle simply spinning away from you.
While holding on to your support of choice, put your best foot on the pedal... then hold on even tighter, and put the other one on. At this point you will probably be supporting yourself almost entirely on your arms and whatever you're leaning on because your legs are all over the place; it's normal! Don't try to go anywhere; just get the hang of sitting there until you feel fairly stable.
Now it's time to try to go somewhere. Going anywhere without holding on is known as "Launching into the abyss"... face a space with no people or objects in it, push yourself away from whatever you were leaning on, and pedal...
Try to do it confidently. Just like a bike, a unicycle is much harder to control if you're going very slowly; you'll probably end up falling sideways. If you have enough speed it will be easier to remain upright. When you start, push yourself off that little bit harder than you think is sensible.
Lean forwards. Just like walking, going forwards involves falling over, delayed indefinitely. You can't just pedal away, as the unicycle will go forwards without you and you'll fall off the back. Lean forwards and try to pedal to keep the unicycle underneath you. A good sign of progress is falling off the front, rather than the back... if you're falling off the back you're being too tentative.
Keep your head up and your back straight. The wheel isn't very interesting to look at. Besides, if you look downwards you'll end up folding at the waist, your weight will move backwards and you'll fall off. Try looking at something in the distance; this will keep you nice and straight, with your weight over the wheel, and give you a target to aim at.
Wave your arms frantically. Some unicycles have a handle at the front of the seat. Don't be tempted to hold on to it; use both hands to wave like a maniac to try to keep your balance.
Try to stay on, no matter what. Unicycling is basically learning how to correct any loss of balance; if you feel yourself going, hang on for dear life; you'll slowly build your ability to correct without thinking about it, and then you've got it cracked.
Don't be afraid of falling off. On a bike, falling off is nasty, mainly because you get tangled in it. On a unicycle however there's nothing to get tangled in; no crossbar to get your leg stuck on, no handlebars to fly over. The majority of falls involve nothing more than the unicycle crashing to the ground and you landing on your feet.
When you can ride a few metres, try to put your weight on the seat, not the pedals. Not just a bit; a lot. All of it. The pedals are for pedalling, not standing on. You don't need to grip the seat with your legs, either; if you sit on it properly it'll stay there. This will take a while to get used to, but it improves your balance and means you won't have aching knees for days. If you feel yourself wobbling from side to side excessively with each pedal stroke, it's probably because you're putting too much weight on the pedals.
Beyond that, it just takes practice. Once you can ride forwards it becomes a lot less frustrating; it takes a while to be able to go anywhere on it, but once you can do a few metres you'll be riding round the block in no time.
When you hear your first shout of "You've lost a wheel!" you'll know you've got it cracked. From then on it's only a matter of time before you'll be looking at mountains thinking "hmmmm..."