The vacation housing market seems to be cooling off.

At first glance, that statement seems hard to prove. After all, the number of mortgage rate locks (when borrowers are guaranteed a specific rate for a certain period of time) on second homes increased 48% in May compared with a year earlier, according to Redfin. But if that sounds pretty robust, consider that since last June, the jump had been greater than 80% each month.

Another Redfin metric, prices of homes in seasonal towns, rose 19% in April compared with 16% in the rest of the U.S. That's down from a spread of as many as 7 percentage points last year.

But prospective second-home buyers shouldn't interpret the data as indicative of some big shakeout and a sign to take action. It's likely to remain a seller's market for vacation homes for the foreseeable future.

While some employees will be returning to the office if they haven't already, many will continue to have some flexibility, or at least more than they used to (or find jobs where they'll be given some). And with limited labor available in seasonal towns, it will take even more time to increase the supply of homes than in the rest of the market.

Owning a second home often comes with headaches and unforeseen expenses. But buying in a market that continues to be so tilted to sellers could prove to be especially disastrous for vacation homeowners. From rushing to buy without understanding local ordinances targeting renters to waiving a home inspection in a destination prone to hurricanes, pitfalls abound. 

Even worse, many seasonal towns have had such a run-up in prices, it's likely buyers may overpay, then be disappointed if they're expecting those eye-popping gains to continue. Think twice before buying in Lake Tahoe, California, where the typical home value is now more than $604,000. As of May, it had the biggest year-over-year jump in home values—28.2%—out of the top 20 metro areas tracked by Zillow with the highest share of homes listed as vacation homes in Census data.

It's a similar story in less expensive areas as well. Home prices have also skyrocketed 28% in East Stroudsburg, Pennsylvania, in the Poconos, where the typical home value is about $227,500. Show Low, Arizona, a picturesque area near the White Mountains, rounds out the top three, with home prices up almost 25%, to $292,150.

Those hoping for a better deal will have to head pretty far afield. Steven McCord, a real estate investor who co-founded Locate Alpha, which provides city and sub-city analyses, says he's surprised at how inflated home values are even in relatively obscure vacation areas in the middle of the country and in the South.

Those with below-average price growth over the past year that offer average prices or lower include Waterloo State Recreation Area in Michigan, Cloudcroft, New Mexico, and the Kerr reservoir area near Clarksville, Virginia, according to McCord.

Still, even if you're able to find a reasonably valued vacation home in this environment, there's a laundry list of things to keep in mind. If you are considering renting it out, make sure you know if there are any local limitations on doing so. Given the frenzied vacation demand, some towns have established new rules (or are in the process of doing so) around what can and can't be done when it comes to renting a home.

Second homes close to the water or in the mountains need more upkeep—from replacing the HVAC and windows more frequently to dealing with pest damage. If you aren’t going to be there all the time, and don’t want to arrive for the weekend to find that flying squirrels have gotten down the chimney and eaten all the window sills (which insurance may not cover) or where the water pipes have burst, you might need to pay someone to check on it.

And finally, be aware that Fannie Mae and Freddie Mac are now subject to a lower cap on how many second-home loans they can buy from mortgage lenders. In turn, that means lenders could be stuck with those loans and will charge extra for that risk.

So if you plan on financing a second home purchase, you may be facing a higher mortgage rate or fees. Yet another reason why buying a vacation home right now may be less than idyllic.

Alexis Leondis is a Bloomberg Opinion columnist covering personal finance. Previously, she oversaw tax coverage for Bloomberg News.