The late Queen Elizabeth was a steady, reassuring presence in the lives of millions, her reign spanning a record 70 of her 96 years.

Widely seen as the woman who defined the modern monarchy, she held her family together through some of its most turbulent chapters — from the immediate aftermath of Princess Diana’s sudden death to the breakdown of her children’s marriages, most notably that of her son and heir King Charles, as well as the fraught and highly public exit of Prince Harry and Meghan Markle,.

But in her final years, one decision tied to the man often described as her favorite son — the disgraced former Prince Andrew — continues to cast a long shadow. Her instinct to protect him amid his links to Jeffrey Epstein may have shaped a move that still perplexes insiders: the reported decision to help fund his estimated $16 million civil settlement in March 2022 with the late Virginia Giuffre, who accused Andrew of sexual abuse when she was a minor — allegations he has denied, settling the case without admitting wrongdoing.

That decision, says Robert Hardman, author of Elizabeth II: In Private. In Public. The Inside Story (out May 19), “will go down as a mistake … and one that has outlasted her,” he tells PEOPLE in this week’s cover story.

Those close to the late monarch acknowledge that the issue of Andrew was one she grappled with deeply — and that it remains a complicated part of her legacy.

“The Queen effectively sacked him and forced him to step back from public life, which clearly he didn’t want to do,” says a close palace source. “I don’t think we should underestimate what it would take for a mother to do that. Throughout, she showed that when it came to the demands of family over the role, the role would win out.”

As the late Queen’s family prepares to lead commemorations for the much-loved monarch, who would have turned 100 on April 21, observers like Hardman say her legacy ultimately looms far larger than the challenges facing the family she left behind.

“Her greatness remains,” Hardman says. “We tend to look at things through the prism of the present — particularly Andrew and Harry. But when you stand back and take in a 70-year reign and a 96-year life, those are important chapters, not the defining elements.”

“She came into a man’s world, held that institution together and handed it on in far better shape than people expected,” he adds.

Adds royal biographer Sally Bedell Smith, author of the Royals Extra Substack: “She was the calm in the face of problems. People knew that they could count on her.”

That steadiness was most visible when the country needed it most.

“In moments of anguish, stress and crisis, she was the glue that kept us all together,” says her former press secretary Ailsa Anderson, recalling the Queen’s “we will meet again” address during the first wave of COVID-19—a message that resonated far beyond Britain.

Anderson believes the royal family can channel her steady resilience to make it through their current turmoil. Anderson tells PEOPLE that their latest troubles “are not going to be [their] downfall.”

“They’re survivors. The monarchy has weathered far worse,” she says.