Under new government plans drivers in England, Scotland and Wales caught using a mobile phone for the first time will automatically receive penalty points.
Previously, motorists in some police force areas could avoid points by taking a remedial driving course. However ministers believe it is not a tough enough measure to deter people from using a hand-held phone while driving.
They have also confirmed plans to raise fines for offences from £100 to £200 and penalty points from three to six.
The scrapping of the driving course option is among several measures announced in a government response to a consultation on punishments for drivers caught using hand-held phones.
The government first announced that it was going to increase fines and double penalty points in September.
The new measures, which are due to take effect next year, follow the jailing last month of lorry driver Tomasz Kroker, who killed a mother and three children while distracted by his phone.
Fine numbers plummet
The number of fines issued for motorists caught using a mobile phone illegally has plummeted by 84% since 2011.
Some 16,900 drivers were handed fixed-penalty notices in England and Wales last year, compared with 123,100 in 2011, Home Office data shows.
Motoring groups believe the decline is due to a 27% fall in the number of full-time dedicated roads policing officers in England and Wales (excluding London) between 2010 and 2015.
Department for Transport figures show that a driver being impaired or distracted by their phone had been a contributory factor in 440 accidents in Britain last year, including 22 which were fatal and 75 classed as serious.
Steve Gooding, director of motoring research charity the RAC Foundation, said: "By ruling out courses and doubling the fine, ministers are reflecting public concern and showing they want to stamp out a potentially lethal activity before it becomes entrenched behaviour for a growing number of drivers."
The measures will not affect Northern Ireland, where drivers are currently given three penalty points and a £60 fine for the offence.
The Department for Infrastructure has said there are no plans to change this, but it "will continue to monitor changes being made in Britain to see what can be learned".