In a 2015 Financial Times interview, Graff Chairman Laurence Graff said about De Beers, “If you want to be successful with diamonds, you have to get as close to them as possible.”

That’s less true now. For the 80-some buyers operating from Belgium to Israel to India, the profit margins on trading diamonds have turned wafer thin or disappeared altogether. Many frustrated sightholders are refusing to buy at current prices, according to people familiar with the matter.

It’s impossible to guess how much money some of the company’s customers—which are almost-exclusively private companies—might be losing. For retailers, like Graff and Tiffany, trading and polishing diamonds is a tiny part of their overall business, and they’re less affected.

But for the many specialist firms, the troubles have been harder to manage. Several have already gone out of business and Eurostar Diamonds, previously one of the biggest buyers of rough diamonds, was put into bankruptcy earlier this year. To be sure, the group has faced previous downturns and De Beers itself reduced the number of sightholders last decade from more than 100.

As its customers struggle, De Beers is also suffering financially. The company’s parent, Anglo American Plc, saw first-half profit drop almost 30% from the diamond business. Sales in the first six months of the year were at least $500 million less than the same period of the previous three years.

Rough diamond prices have declined about 6% this year, while polished stones are about 1% lower, according to data from Polishedprices.com.

Sightholders face shrinking profit margins from buying from De Beers. The built-in discount they receive has been about 3% in the past year. In some cases, it’s been whittled down to as little as 1% or they’ve paid more than they can sell them for in the secondary market, according to customers and others in the industry.

The narrow margins means profit on the boxes is wiped out after accounting for a 1.5% value added service charge payable to De Beers and broker fees of about 0.5% to 1%, they said.

It’s a sizable shift from the end of the last decade, when customers could consistently expect a discount of more than 5%, according to the people.

De Beers has gone to unusual lengths to support its struggling buyers. The company this year loosened the rules for sightholders by letting them lower their annual quotas and defer purchases. It also says the pricing allows customers to make good-enough profits after averaging out market booms and downturns.