Two air strikes targeted the ministry late on Friday and plumes of smoke could be seen billowing from the building, the rebel media network Al Masirah quoted witnesses as saying.
This comes as Saudi Arabia recently intercepted a ballistic missile fired from Yemen near the capital, Riyadh.
The European Commissioner for humanitarian aid called on the Saudi-led coalition to allow aid to reach the Yemeni population.
The United Nations and more than 20 aid groups said on November 9 that the Saudi-led coalition's tightening of a blockade on war-torn Yemen could bring millions of people closer to "starvation and death".More news: Leak reveals OnePlus 5T ahead of November 16th announcement
A Saudi-led, USA -backed coalition allied with Yemen's internationally recognized government has been at war with the rebels since March 2015.
The war in Yemen, the region's poorest country, started in 2014 after Houthi rebels seized control of Sanaa and began pushing south towards the country's third-biggest city Aden.
"Washington aids the forces of aggression, politically and militarily, and has pushed them into risky pitfalls", Houthi spokesman Muhammad Abdul Salam said on the group's Al-Masirah TV channel.
Should Monday's blockade - that saw the coalition force the closure of all of Yemen's borders - continue, says the United Nations, it would bring millions of people closer to "starvation and death".More news: Sherwood students honor vets through annual program
Commenting about the recent Yemen missile attack targeting the global airport in Riyadh, he said the missile command of the Yemeni army and Popular Committees demonstrated it capability to target the capitals of all hostile countries.
Iran denies it supplies arms to the Houthis but has said the missile was a reaction to Saudi "aggression".
On Sunday, the head of the Houthi supreme revolutionary committee, Mohammed Ali Al Houthi, wrote on his Facebook page that the Houthis would target oil installations in Saudi Arabia with missiles if the coalition attacked Hodeidah.
Three years into a brutal conflict, Yemen depends on imports - amounting to up to 90 per cent of its daily needs - and millions in the country are being kept alive by humanitarian aid.More news: Stephen King and Chrissy Teigen Lead the 280 Character Rebellion on Twitter
The Yemeni government-held southern port of Aden reopened on Wednesday, but ports in Houthi-held areas are still closed.