I recently thought of teaching my own future children to count to 1023 in binary using this same technique. Unfortunately, I also realized the difficulty in doing so: counting in binary requires either knowledge of multiplication and addition in order to compute the numerical value of a given arrangement of fingers, or the ability to recognize on sight every possible arrangement of fingers (although with just one hand, this is limited to 31 possible numbers instead of 1023, so it's a bit more managable).
There's also the conceptual issue. A preschooler learns to count from one to ten on his/her fingers because they can conceptually recognize four fingers as "four", the same way they identify four apples or four blocks as "four". Identifying "four" as middle-finger-up, all-other-fingers-down does nothing to help the preschooler to recognize "four" of any other things in real life.
However, it does have potential to simplify basic math with older children. Counting in binary on your fingers means that you can calculate sums and differences larger than ten, multiply quickly by two and powers of two, and of course have a leg up on the rest of the class if and when you begin a career in circuit design.