And now the British actor is reportedly thinking about moving back to his native Blighty to further distance himself from the scandal.
"Robert is thinking long and hard about returning to London," a source close to the actor said.
"He's selling his home in Los Angeles, not only because it holds bad memories, but because he wants a fresh start.
"He never fell in love with Los Angeles and has always longed for home. When Kristen was shooting Snow White and the Huntsman in London, he made his excuses to visit her as much as he could – he missed it so much.
"Despite making friends in America, Robert longs to just have a pint with the mates he grew up with in a local west London pub. He's just a regular guy.
"He is set to make up his mind very soon – his family hasn't put pressure on him to return – but he misses them very much and they've said they'd welcome him back with open arms."
Rob recently listed the Los Feliz, California pad he shared with Kristen for $6.3 million because "there are too many memories" there.
The former couple are apparently now back on speaking terms and although Rob doesn't hate Kristen and actually still loves her he is still bemused and "doesn't get why she did it."
Meanwhile, Kristen is said to be "close to a total meltdown" and "stays up all night crying."
For access to all our exclusive celebrity videos and interviews – Subscribe on YouTube!