Oliver, who had been the natural resources minister, replaces Jim Flaherty, who has decided to resign from cabinet.
Rickford has been promoted from the role of minister of state to take over the Natural Resources portfolio.
Ed Holder, MP for London West, replaces Rickford as minister of state for science and technology.
“I just named Joe Oliver Canada’s new finance minister,” Harper tweeted as the new ministers were sworn in at Rideau Hall. “He will continue to strengthen the economy & balance the budget by 2015.”
The 73-year-old Oliver is widely respected within the Harper cabinet and among Conservatives, and is well known on Bay Street, having spent decades working there as an investment banker.
He went on to become executive director of the Ontario Securities Commission and then president of the Investment Dealers Association of Canada.
Oliver’s sometimes gruff and often partisan demeanour is far from that of the jocular Flaherty.
Rickford, 46, retains responsibility for economic development in Northern Ontario. MP for Kenora, Rickford was first elected in 2008.
He will become the government’s point man on the tricky Keystone XL pipeline project, which is still awaiting approval in the U.S.
Holder, 59, is an insurance broker first elected to Parliament in 2008.