It certainly wasn't originally true. When the US constitution was written voting rights were under the control of the individual states.
https://en.wikipedia/wiki/Timeline_of_voting_rights_in_the_United_States The process of getting to universal suffrage took a while, and doesn't seem to be completely accepted even now.
The Australian constitution is a much more recent document.
In Australia you lose your right to vote while you are in prison if you have been sentenced to prison for a term longer than three years. There was an attempt to disenfranchise all prisoners in 2006, but it got reversed in the high court in 2007.
Gerrymandering is probably a more serious problem. Somebody setting up gerrymandered electoral boundaries wants their party to win ans many electorates as possible by the smallest possible margin, while the other party will have huge winning margins in the few electorate where they do win.
If lots of electorate are won by tiny margins, one has to suspect gerrymandering.