During the time in South Trenton, Lee continued her correspondence and even initiated occasional visits with AJ in New York. He now had career working in a major museum, first as a guard then as an art restorer. He was also starting to sell his own artwork. Not much is known about the exact nature of their relationship except that it encompassed the emotional, intellectual and physical. There is some evidence that it was not symmetrical; that Lee was more deeply involved than AJ. After the accident, the intensity of the relationship, at least from Lee's perspective, increased.

The family moved again in 1951, once again crossing the Delaware, out in the country in Middletown Township, Lower Bucks County. The homestead had 2 acres of land, a large house with four bedrooms and a "walk-in" fireplace. There was a foyer between the two ends of the house which Lee commandeered as her special hobby workplace. To help defray the cost, Lee's brother Joe and his wife Gloria moved in with them. The bowling alley had been sold and Bob was now employed fulltime selling structural steel. Gary was starting seventh grade in a new 7-12 high school and Robin was anticipating kindergarten the next year.

Lee had been experiencing severe angst about the relationship with AJ; she was feeling rejected, likely because she was pressing AJ for some kind of commitment. At some point, she made one last visit to New York. It is only from the fall out that one can imagine what occurred: she was fully rejected, the unrequited nature of her love made explicit, the relationship rendered, and she exiled to her family in PA. Shortly after that, now 31 years old, Lee sequestered herself in her foyer room and began to write.