Prosecutors say he stole an estimated £117m from the oil-rich state.
This is the first time that money recovered from criminals will be returned to Nigeria since an agreement was signed in 2016, the UK authorities say.
The UK and Nigerian governments signed an agreement on Tuesday for the £4.2m recovered by UK agencies to be sent back to Nigeria for infrastructure projects, including the Lagos to Ibadan Expressway, the Abuja to Kano road and the second Niger Bridge.
James Ibori went from petty thief to Nigerian state governor to convicted money launderer.
He went to the UK in the 1980s and worked as a cashier at a DIY store in London.
Ibori was convicted in 1991 for stealing from the store but then returned to Nigeria and got involved in politics.
When he ran for Delta State governor, he lied about his date of birth to hide his UK conviction – which would have prevented him standing for office.
He became governor in 1999 and soon began taking money from state coffers. Delta State is the source of much of Nigeria’s oil.