It's usually taken me ~4-5 days to get back to normal after quitting caffeine. It degrades with a half-life, so the last of it takes longer to clear than the bulk, and part of the recovery is a few days of good sleep.

I haven't successfully stayed off it for a long time since the kids were born. Maybe I'll try again when I go to the UK.