Stanton is correct when he points out you're attempting to use a very narrow and selective view of the law. Regardless, lets lay it out for you...
The Constitution Act of Canada (Commonly called refered to as "The Charter") is what gives individuals their legal rights. Section 6 gives you the right to remain in Canada or to move from province to province. Section 6(3) however puts limits on those rights:
(3) The rights specified in subsection (2) are subject to
(a) any laws or practices of general application in force in a province other than those that discriminate among persons primarily on the basis of province of present or previous residence; and
(b) any laws providing for reasonable residency requirements as a qualification for the receipt of publicly provided social services.
Another section that may apply is Section 9, which focuses on arbitrary detention. While I'm sure you would argue that by not being able to drive that it is impossible for you to excersize your right to move from Province to Province, there is nothing anywhere that says you must be able to excersize your rights in whatever fashion is most convenient to you. Therefore, take a bus, plane, horse, bicycle, skateboard, roller-blades, snowshoes, skis, or simply walk. Without a valid driver's license, you're not allowed to drive.
The law for driver's licenses and the fees behind them are to protect all people on the roads and sidewalks. The rules ensure people know how to drive safely, even if they choose not too. And the money helps to pay for maintenance to those roads, emergency services for those injured on the roads, and for enforcement to reduce those injuries.
So feel free to test your argument in court. My guess is you'll only annoy the judge and the fine will be increased, meaning that more money is collected for keeping our roads safer.