John Phillips, 39, built the four chalets in his garden in the Gower, South Wales, in 2022, to rent out on Airbnb for up to £185 a night.
But he did not have the correct planning permission for the pods in Landimore which overlook the Wales Coast Path and Loughor Estuary.
Swansea Council turned down his retrospective planning application in July last year and sent him an enforcement notice four months later.
It said he had to stop renting them and demolish them and tidy up the land within six months.
John then appealed the refusal decision and the enforcement notice but now a government-appointed planning inspector has dismissed both appeals.
So now he finally has to take them down.
But he was granted 12 months, rather than the original six, to comply with the order to demolish and restore the site.
The inspector, Richard Duggan, accepted more time was needed to cancel bookings at the high demand chalets.
John, who says he intends to fight the case in court, called the order “madness”.
He said: “It’s really unfair – it’s one rule for us and another for everyone else.
“They [the council] have just awarded a house in our village permission to build a ten room Airbnb.
John, who rents out the chalets with his wife Kerrie Garrett, 34, and lives on the site with daughter Darcy, four, says he doesn’t know what they will do if they have to bulldoze the holiday homes.
“We won’t be able to afford to pay our bills if this happens,” he said.
“It pays our living – it’s our full-time job.
“We haven’t looked at price for demolishing it. We are going to fight this every step of the way.
“We have hired a planning consultant and are currently working on an appeal.”
John move into the property three years ago, but believes his family are being treated differently because they are not locals.
“We’re from Swansea and they don’t like someone from outside the Gower coming in,” he said.
“It seems like there is a hidden vendetta against us.
“It’s all who you know down here.”
Planning inspector Richard Duggan said the chalets decking and hot tubs were visible from the coast path.
He said: “The view from the public footpath, albeit only for approximately 100m, is valuable as part of the experience of walking within the AONB and the development seriously harms that view.”
Swansea Council said: “We welcome the decision of the planning inspector in supporting the previous enforcement actions taken by the council in relation to this unauthorised development.
“Hopefully the outcome of this case will send out a clear message that anyone considering development in the city should make sure they do so with the proper planning permissions in place.
“This is particularly important in terms of development on Gower because of its protected status where additional planning policies are in place to ensure development is appropriate for Gower and in the right location.”
