Robert Downey Jr. hasn't always been an "Iron Man."

There was a time in the 1990s when the actor was struggling with addiction and in a great deal of legal trouble.

During a recent conversation for Dax Shepard's "Armchair Expert" podcast, Downey talked about being incarcerated in 1999, which he likened to "being sent to a distant planet where there is no way home until the planets align."

"You could just feel the evil in the air, and that was no trouble at all because it was kind of like just being in a really bad neighborhood," he said. "There was no opportunity there. There was only threats."

Downey Jr.'s brushes with the law have been well documented.

He was arrested in 1996 for possession of heroin, cocaine and an unloaded .357-caliber Magnum, given three years of probation and required to undergo mandatory drug testing.

A year later, he skipped a court-ordered drug test and spent nearly four months in the Los Angeles County jail.

He skipped another drug test and was sentenced to three years of prison in 1999. Downey served 15 months in state prison in Corcoran, California.

Four months after his release, he was arrested on Thanksgiving weekend for alleged cocaine and Valium possession and being under the influence of drugs. The valium charge was eventually reduced to a misdemeanor, he pleaded no contest to the remaining charges and avoided more jail time.

He also was arrested in April of 2000 after Los Angeles police found him wandering in an alleyway.

Long since sober and successful, Downey Jr. described during the podcast how deeply troubled he was at the time.

"I'm gonna try to give you the flashcards," he said. "I'm in court, I'm being over-sentenced by an angry judge, and at some point he said something in Latin. I thought he was casting a spell on me."

According to the Marvel star, two weeks later he was in Delano, "a receiving center where they decide where you're going to go" that he said was "arguably the most dangerous place I've ever been in my life because nobody is designated."

"I remember walking out at one point when I hopped out of my cell to go to the shower, by the way, this would be the best soundbite, and I didn't know it but I was a little spun out and I had my underwear on backwards," Downey Jr. recalled. "I remember eliciting some strong chuckles and jeers from my fellow inmates."

And while he described being incarcerated as "the worst thing that happened to me," the Downey said he eventually was able to adjust and survive.

"We are programmed to within a short amount of time be able to adjust to things that are seemingly impossible," he said.

Downey is the host of "Downey's Dream Cars," a docuseries debuting Thursday on Max, which is owned by CNNs parent company.