Rishi Sunak hit out at “top-down targets” on housebuilding as Downing Street insisted he stands by the Conservative manifesto commitment to build 300,000 homes a year by the mid-2020s.

Housing Secretary Michael Gove, delivering a speech on planning reforms in London on Monday, expressed his commitment to the target, but did not say when it would be reached.

Ministers are well short of the figure after the Prime Minister last year made it advisory rather than mandatory as he looked to see off a potential backbench rebellion.

There were signs of more local Tory opposition to his housing strategy as Conservative MP for South Cambridgeshire Anthony Browne vowed to “do everything” to stop the “nonsense plans to impose mass housebuilding on Cambridge”.

Responding to the criticism, Mr Sunak stressed the importance of taking “local communities along with you” as he said Westminster politicians should not “ride roughshod over those views, impose top-down targets, carpet over the countryside”.

Asked whether the manifesto pledge of 300,000 new homes a year still stands, the Prime Minister told reporters at a housing development site in the West Midlands: “We are making progress, I’m proud of that progress and we’re not stopping there.

“But we’ve got to do it in the right way, I don’t want to concrete over the countryside, that’s something that is very special about Britain.

“We’re making it easier for people to expand homes upwards and outwards, we’re making it easier to build on brownfield sites with more investment, and we’re investing in the planning system.”

Although Mr Sunak did not explicitly say he remains committed to the 2019 pledge, his official spokesman later insisted he does while emphasising the Government is on course to meet its target of building one million homes between 2019 and 2024.

“Obviously we want to continue the progress towards that 300,000 per year commitment, and obviously we are on track for delivering a million more by the end of the Parliament,” the official told reporters.

It comes two weeks after the cross-party Levelling Up, Housing and Communities Committee warned that Tory ministers are unlikely to deliver it and said Mr Sunak’s decision to drop the compulsory target is “already having a damaging impact on efforts to increase the building of new homes”.

Mr Gove said after a speech in King’s Cross: “The 300,000 target by the middle of this decade is one I completely stand by.”

Asked when the Government is expected to hit it, he said: “As soon as we possibly can.”

He said rampant inflation makes “delivering against that target more difficult” but that he is “confident we are on a trajectory to reach that 300,000 target”.

Labour demanded the Government publish any impact assessment it carried out into the move to ditch the compulsory housebuilding target.

Shadow housing secretary Lisa Nandy said: “The Government has never met its target to build 300,000 homes a year and today’s announcement doesn’t come close to matching the scale of ambition needed to fix the housing crisis.

“Families and young people desperate to get on the housing ladder deserve to know what assessment the Government made of Rishi Sunak’s reckless decision to scrap housing targets, which has pushed housebuilding off a cliff.”

In his speech, the Housing Secretary set out measures including on leasehold reform, simplifying planning procedures, expanding planning capacity, and regenerating and reviving inner cities.

A new “inner city renaissance” is the most important component of the strategy, Mr Gove said, as he argued for “using all of the levers that we have to promote urban regeneration rather than swallowing up virgin land”.

He said there is a “direct contrast” with Labour on the issue, with the Opposition pledging to make it easier to build on unsightly parts of the green belt.

The proposals include developing a new urban quarter in Cambridge with space for homes, art facilities, laboratories and green areas.

But Mr Browne lashed out at the plan, tweeting: “I will do everything I can to stop the Government’s nonsense plans to impose mass housebuilding on Cambridge, where all major developments are now blocked by the Environment Agency because we have quite literally run out of water. Our streams, rivers and ponds already run dry.”

In response, Mr Gove said he believes his colleagues will be won over by the detail of the plans.

The senior Tory said: “It will be the case that, I’m sure, Conservative backbenchers and others, once they have a chance to look at our plans, will realise that this is in the national interest and that’s why we’re acting.”

Concerns have been raised that Mr Gove’s proposals to slash red tape to pave the way for more conversions of shops and takeaways into houses could lead to substandard housing.

Polly Neate, chief executive of the charity Shelter, described his speech as “a real mixed bag”.

“We need proper investment to build much-needed genuinely affordable homes, not more piecemeal reform,” she said.

“Converting takeaways and shops into homes and restricting building to city centres won’t help. It could risk creating poor quality, unsafe homes that cause more harm than good.

“When we are losing more social housing than we build, the Government must work with councils to deliver the quality homes local communities across the country need.

“The Secretary of State clearly agrees these homes are essential, so the Government should put its money where its mouth is and get on with building a new generation of social homes.”

Mr Gove argued for “gentle densification” of inner cities, saying the urban sprawl of the past has hurt productivity.

But Ant Breach, senior analyst at Centre for Cities, said densification will not solve every problem.

“Cities will need to expand outwards too, and this means green belt reform is essential. Only by building both up and out of cities will we start to close the gap on England’s 4.3 million missing homes.”

Mick Platt, director of the Residential Freehold Association, said the Government must “think carefully” about the challenges associated with high-density housing.

“Managing large, complex apartment buildings is not straightforward and the Government’s leasehold reform agenda risks pushing this responsibility on to residents, many of whom do not want it.”