In the decade since hedge fund billionaire John Paulson took a grand gamble on Puerto Rico, he’s faced the wrath of the markets and mother nature.

He’s navigated hurricanes, earthquakes, the pandemic and the largest municipal bankruptcy in US history to amass a portfolio of luxury hotels and resorts, high-end office blocks, and auto dealerships catering to the island’s rich.

Now, just a few months after breaking ground on one of San Juan’s tallest and most exclusive residential towers, Paulson is facing a new wave of threats: lawsuits that strike at the heart of his Caribbean empire.

In New York, he’s caught in a bitter divorce, with his ex-wife alleging that he manipulated the value of one of his Puerto Rican properties to drain at least $10 million from a trust for his kids.

On the island, he’s facing lawsuits by his longtime business partner, Fahad Ghaffar, who claims Paulson is trying to cheat him out of his share in their Puerto Rico ventures.

Paulson, who declined to be interviewed, is pushing back. He denies his ex-wife’s accusations and has filed a racketeering and corruption complaint against Ghaffar and his family that alleges they embezzled millions from the Puerto Rico enterprise, “betraying Paulson’s trust and biting the hand that had fed them.”

Read More: Paulson Divorce Fight Escalates Over ‘Sham’ Luxury Condo

The stakes are high. Paulson’s Puerto Rico portfolio could be worth as much as $1 billion, according to Bloomberg calculations, and according to Ghaffar will make another $1 billion over the next four years. Tourism is booming as the island’s siren song of sun and sand lures high-end visitors to places like the $2,000-a-night St. Regis Bahia Beach Resort, which Paulson acquired a decade ago. Meanwhile, the influx of $120 billion in federal reconstruction funds in the wake of Hurricane Maria in 2017 is adding fuel to the economic fire.

Outside of Puerto Rico, Paulson’s footprint has shrunk dramatically. Shorting subprime mortgages before the market collapsed in 2007 helped swell assets at his hedge fund Paulson & Co. to more than $38 billion in 2011. But wrong-way bets on gold, the US recovery and the European debt crisis hurt returns, and by the end of the decade assets under management fell to less than $10 billion. In 2020 he returned investor funds and converted to a family office.

Since 2020, the value of Paulson’s publicly-traded equities has shrunk from $4.1 billion to $1.1 billion at the start of this year. Much of that drop occurred in 2022, when the family office sold stock worth hundreds of millions amid losses in most of its biggest positions.

While equities may only account for a portion of the activities, Paulson’s net worth is now $4.4 billion after peaking at $13.8 billion in 2014, according to the Bloomberg Billionaires Index. A complaint filed by Ghaffar Thursday says Paulson’s net worth has sunk to less than $500 million, although that might not include assets in family trusts.

Paulson’s lawyers declined to comment on his net worth.

As Paulson’s Puerto Rico portfolio has gained in importance, so have the potential repercussions of Ghaffar’s lawsuits.

Wearing a black blazer over a gray dress shirt at a recent weekday meeting, Ghaffar looks younger than his 39 years.

After graduating from the University of Virginia with a degree in economics and finance, he worked for a fund at UBS Group AG that dealt with toxic assets in the wake of the financial crisis. He left UBS in 2011 to start investing on his own and met Paulson in 2013. At the time, the billionaire was looking to buy the St. Regis resort and Ghaffar says he was considering a stake in a Trump-branded golf course at nearby Coco Beach that eventually went bankrupt.

Soon they were working and investing together. Ghaffar says he has a 44% equity stake in the Vanderbilt Residences, the new 22-story luxury tower that’s key to Paulson’s future growth. In addition, he owns a portion of the Condado Vanderbilt Hotel and La Concha Renaissance Resort, both in the heart of San Juan’s tourist district, as well as stakes in other Paulson assets.

“John and I shared a beautiful relationship for 10 years. We added a lot of value. We created a tremendous amount of jobs for the Puerto Rican people,” he said in an interview in San Juan. “However, we’re both now at the point where I don’t believe we trust each other. And the best way forward for us would be to split the assets in an equitable manner.”

A Paulson representative said Ghaffar is exaggerating his role. “He was just an employee with fractional, if any, ownership stakes,” they said.

An Oct. 16 lawsuit filed by Paulson says Ghaffar was “an unemployed small-time commercial real estate investor in Tampa, Florida who was floundering to make deals” when he approached the billionaire in 2013. He was hired as a junior analyst and eventually promoted to oversee all of Paulson’s Puerto Rico investments.

Once there, the lawsuit alleges that Ghaffar used family members and shell companies to siphon millions from the businesses to support an increasingly lavish lifestyle: six-figure shopping sprees at Louis Vuitton and Chanel, more than $600,000 in private jet travel and $20,000 dropped during a single night of partying at a Las Vegas night club.

Paulson is asking for damages of $190 million.

Ghaffar’s lawyer, Martin Russo, called Paulson’s suit a “publicity stunt.”

“We look forward to dismantling their lawsuit and vindicating Mr. Ghaffar and his family,” he said.

Read More: IRS Probe of 100 Wealthy Americans Gets Boost From Puerto Rico

For years, the investors appeared to be a powerful team: Ghaffar, a native of Pakistan, living on the island and managing daily affairs, while the 67-year-old Paulson — one of the most prominent names in finance — remained in New York. They were often seen as the face of the investor community on the island, their luxury hotels and resorts providing a backdrop for political gatherings, celebrity sightings and corporate power lunches.

But behind the scenes, tensions were building.

In a lawsuit filed Thursday, Ghaffar said he “worked tirelessly” for Paulson for a decade and was entitled to a percentage of the upside. But Paulson was “jealous and hostile” as Ghaffar stood to make almost $300 million in the next four years with their Puerto Rico investments, according to the suit. Not long after a meeting in June, the billionaire fired Ghaffar, who is asking for damages of at least $237 million.

Paulson’s lawyers have argued, in a separate lawsuit, that Ghaffar was finally fired when he “went on a two-month vacation on his recently purchased yacht in the Mediterranean” and his years of wrongdoing came to light.

Ghaffar “was terminated for cause and no longer entitled to any compensation,” Paulson’s lawyers, Terrence and Darren Oved of Oved & Oved LLP, said in a statement.

The battle with Ghaffar comes as Paulson is already caught in a contentious divorce with his ex-wife, Jenica Paulson. Among her allegations is that Paulson bought a penthouse at the St. Regis from one of his family trusts at an artificially low price, effectively allowing him to siphon $10 million from the entity he wasn’t entitled to.

She’s asking for at least $1 billion in damages.

Jim Smith, a partner at Blank Rome who represents Paulson, called the allegations “absurd.”

“These fictitious claims are just another attempt to deflect attention from her effort to strip her children of their inheritance,” he said.

Read More: Puerto Rico Aims to Woo NY and Put Bankruptcy in Rearview

Paulson’s legal woes have captivated Puerto Rico’s tight-knit business community, where the billionaire has helped reshape the tourism industry, fueled a building boom and created thousands of jobs.

In 2014, when Paulson first invested in the Vanderbilt Hotel, the 104-year-old building had been abandoned for almost two decades — a fitting metaphor for the US territory of 3.2 million people that was going through a deep recession.

“John started investing in Puerto Rico when the situation was dire,” said Jose Carrion III, an insurance executive and the former head of the island’s powerful financial oversight board. “He became a spokesperson, a symbol, of people willing to put their money where their mouth is in regards to investing on the island.”

Whether Paulson’s run of good luck on the Isla del Encanto continues may depend on the outcome of the trials. But neither Ghaffar nor Paulson say they’re willing to step away.

Ghaffar says he’s moved his mother, sister and brother to Puerto Rico. His wife is from Puerto Rico and his daughter was born there. “I don’t have another home elsewhere,” he said. “I’m as local as an outsider could be.”

Paulson recently appointed his nephew, Rafael Cedeño Paulson, as executive vice president to oversee Puerto Rico affairs.

“We have a great team of employees,” Paulson’s representatives said, “and our commitment to the island has never wavered.”