Sports clubs represent the ultimate asset trophy, bringing with them prestige and many times profits. But for a lot of the billionaire buyers, the reason for owning a team may come down to one simple fact: It’s a lot more fun than sitting on a pile of money.

That's even with the risk of being looked upon as a villain by millions of sports fans.

“There is broad risk exposure in acquiring a team and yet the thrill of potentially winning may overshadow potential losses,” said Bennett Cherry, an associate professor of entrepreneurship at California State University in San Marcos. “For owners, trophies don't just come by winning championships. Trophies can also be had simply by showing your colleagues that you own a sports team,”

Observers note sports team ownership is not all glamour and prestige. It also brings vast responsibility—stewardship over what are virtual public institutions that are passionately followed by millions. When those fans don’t like the way their teams are being run, their ire is often directed to the person at the top.

“Most are risk taker, type-A personalities,” said Lee Hutton, a Minneapolis-based attorney who has dealt with sports owners as a counsel to NFL players. “They have to keep fans, athletes and city officials happy."

Sports teams—whether they are baseball, basketball, soccer or football teams—can bring back astronomical returns over the long term, but many owners don’t seem to buy with the intention of selling, according to industry observers.

George Steinbrenner, for example, bought the New York Yankees baseball team in 1973 for about $9 million. The gregarious owner, who was known for not sparing any costs in putting together star-studded lneups, watched the value of the team soar to more than $1 billion over the next three decades. Yet he never put the team up for sale and held it at the time of his death in 2010. The team remains under the ownership of the Steinbrenner family.

"The Steinbrenners are having too much fun to sell the Yankees because New York is such a good market. Fun is the ego of owning the sport and the power of being in the owner’s box during games,” said Bob Weiner, an anti-doping activist who sits on the board of the U.S. Track and Field National Masters. “TheYankees are gold standard because of Yankee stadium and the New York media."

Other wealthy individuals who have benefitted from owning a team include George Soros, Paul Allen and the Tisch family. Last year, Mexican business tycoon Carlos Slim purchased a 30-percent stake in Grupo Pachuca, whose holdings include first division Mexican football clubs Pachuca and Leon.

About 12 out of 20 who own sports teams around the world have been owners for less than 10 years, according to Wealth-X.

“Most owners buy and hold because they are not buying for the team itself. They are buying for ancillary revenues. When it comes to selling the teams, you have to wait for the ancillary businesses to mature and that doesn't happen overnight. It can be a 10-and-20-year investment,” Hutton told Private Wealth.

While selling may be out of the question, moving is often one way that owners try to profit from the team they own. It is also a good way to become an object of scorn by entire cities or regions.

“When owners move a team it's because they lost more money than they can tolerate and they are looking for a better market,” said Weiner.

In the 1990s, fans vilified the owners of the Houston Oilers NFL football after they moved the team to Nashville, Tenn., renaming the team the Tennessee Titans.

Art Modell, who died in 2012, was similarly loathed by fans after he took his Cleveland Browns—an NFL football franchise that had existed in the city since 1945—and moved them to Baltimore in 1996. Modell’s team, renamed the Baltimore Ravens after a court ruled that the Browns rightfully belonged to Cleveland, went on to win a Super Bowl four years later.

“It's difficult for owners to buy a team and then move it because it competes with the owner’s investment goals and the local cultural identity of the team,” Hutton said. “They don't often match.”