According to royal author Carolyn Durand, the hardest part of stepping away from his senior royal duties was having to give up his honorary military titles and his role as Commonwealth Youth Ambassador.
"It stung, and even today he has not been able to move on from that," Durand said.
The author further noted that Harry's denied request for a wreath to be laid on his behalf at the Cenotaph on Remembrance Sunday, "hurt as much as the day his honours were taken away."
After serving ten years in the army, the red-headed prince's military titles have been put on hold; he is not allowed to take on any roles using his titles since his royal departure last year.
Luckily, Harry, 36, may have a chance to reclaim his titles.
The Queen is due to review Harry and Meghan Markle's Megxit deal — which first came about last January, and is set to end on March 31 — which will solidify where her grandson stands.
"Harry wants it both ways," a source told OK!. "He wants back all his military roles and wants to be treated as a senior member of the family at all the official events, sitting in the front row next to his father, Charles, and brother, William.
"But he also wants to live in America and cut business deals with media companies," the source added.
As OK! previously reported, the Prince and Meghan will return over the pond to attend Her Majesty's "Trooping of the Color" parade in honor of her 95th birthday on June 12.
The couple is set to meet with the senior royals on a Zoom conference in January to discuss extending their deal for another 12 months, before they head back to the U.K. for the first time since their royal fallout.
Harry and Meghan, 39, stepped away from their duties last March so they could be financially independent and rebrand themselves as Hollywood stars.
However, the Duke and Duchess of Sussex aren't ready to shut the door on their royal lives, as they hope to carry on as non-working royals in California while they make their millions in new deals with Spotify and Netflix.