It is a misconception and a misplacement of respect
to refer to all Australians
Only Australian soldiers are honoured with that term of endearment in their native country. The term comes about from the spectacular service that the Australian armed forces gave on the Western Front in the First World War, when, being at the bottom of the pecking order for the British forces, were required to dig most of the trenches in the European theatre.
In modern times, "Digger" is only ever applied to surviving servicemen from WW1 and occasionally in a group sense for servicemen from WW2.