Apparently, only some parts of this speech were improvised, not the whole thing. Rutger Hauer has this to say about the tears in rain speech on his official site: (italics mine)
Originally it was a bit longer, like a half-page of dialogue. So I said to Ridley the night before we shot it, 'This is way too long. If the batteries go, the guy goes. He has not time to say good-bye, except maybe to briefly talk about things he's seen' Life is short - boom! I truly felt that the ending of this picture should be done very quickly, I mean, we'd already seen this opera of dying replicants; I didn't think the audience would stand another protracted death scene. So I said to Ridley, 'Let's do it very fast, and do it as simply and profoundly as possible. But also, let Batty be a wiseguy for a second'. Ridley said, 'Yes, I like it'. So when we filmed that speech, I cut a little bit out of the opening and then improvised these closing lines, 'All those moments will be lost in time. Like tears in rain. Time to die'.

But you know, everyone always writes about me and that speech, and ignores the screenwriter. I thought David Peoples, the man who wrote that version of Batty's soliloquy, really did a beautiful job. I mean, I loved those images he came up with -'c-beams glittering near the Tannhauser gate, attack ships on fire off the shoulder of Orion'. I thought they were really interesting, even if you didn't understand them. The whole idea there, is that once he stops talking, the dove flies. You never really see the moment of Batty's death, the dove says it for him.