Perry recalled how during their group dinner party, the actress "wasn’t interested in [him] at all," and as soon as they arrived at the bash, Diaz got "stoned."
Things took an even more awkward turn when the group played Pictionary, as he claimed the mom-of-one "accidentally" hit him square in the face even though she was targeting his shoulder. Confused and surprised, Perry responded with, "Are you f***ing kidding me?"
The TV icon dished on other former flames in his book as well, divulging that he once made out with Valerie Bertinelli, 62, while her husband at the time, Eddie Van Halen, was passed out next to them.
Perry insisted he had "real feelings" for the Food Network star, though after their hookup, she played it off as if nothing happened. "I quickly got the hint," he recalled, admitting he had "many a tearful night" over the ordeal. "But inside I was devastated."
- Swing & Miss! Matthew Perry Once Awkwardly Told Jennifer Aniston 'We Can't Just Be Friends' After She Rejected Him
- Matthew Perry Reveals Secret Relationship With Julia Roberts & How He Convinced Her To Be On 'Friends' In New Memoir
- Matthew Perry's Previous Relationships: 15 Women He Was Linked to Over the Years
The Studio 60 on the Sunset Strip alum has also dated Neve Campbell and Lizzy Caplan, but his most famous Hollywood ex is Julia Roberts, 54. He confessed in his book that their high-profile romance was "too much for" him and brought about feelings of insecurity since she was one of the biggest movie starlets of the moment.
"I had been constantly certain that she was going to break up with me. Why would she not? I was not enough; I could never be enough; I was broken, bent, unlovable," he wrote in his book. "So instead of facing the inevitable agony of losing her, I broke up with the beautiful and brilliant Julia Roberts."