The first 45,000 letters have gone out on Tuesday.
About 5.5 million will be paid back an average of £419 ($US644), while 1.4 million will have to pay about £1,500 ($US2,300).
Payback will start next April when State Benefit will be cut, National Insurance will rise and VAT will go up to 20%.
There are only 24 million people working, according to Sky News, so, if we do our sums right, that’s nearly 1 in 4 taxpayers who have had their tax estimated wrongly.
Experts advise people who have legitimately filled in their tax returns to contact the tax office saying they have done no wrong and should therefore not have to pay.
The Pay As You Go (PAYE) system for taking tax from workers has grown to six times its original size.
Usually its end-of-year cross-referencing is done manually on a one-by-one basis but a computer system was introduced which proved insensitive.
A similar situation happened in May when 100,000 people paid too much tax because of a computer blunder.
John Whiting, tax policy director at the Chartered Institute of Taxation, according to the Telegraph, said at the time: "Huge numbers of people were given the wrong tax codes by HMRC's new computer system.
"Revenue staff have worked incredibly hard to review three million records in March and April. But this situation should never have arisen in the first place.
"The Revenue's contact centres have been struggling to cope. HMRC bosses need to take a close look at whether they are adequately resourced for the future."
Mr. Whiting joined former Treasury Minister Michael Jack in the Treasury's Office of Tax Simplification in July. They are charged with finding ways to reduce the complexity in the tax system for small businesses and propose alternatives.
