The quake, which violently shook buildings in the capital Taipei on Sunday, sent people running into the streets and was felt in Hong Kong, more than 700 kilometers (435 miles) away.
After the National Fire Agency reported an initial death toll of two people, rescuers found the body of a man previously listed as missing, and a victim injured by falling rocks later died in hospital.
Three of the victims were killed in central Nantou county, the epicenter of the quake whose magnitude Taiwan’s Seismology Center measured at 6.3, while the US Geological Survey put it at 6.2.
Tremors caused widespread landslides in the mountainous county, as well as cracks in roads and along the walls of houses.
Television stations reported panic-stricken shoppers running out of a 12-storey department store in the central city of Taichung.
Nantou county was the epicenter of a 7.6 magnitude quake in September 21, 1999, that killed around 2,400 people in the deadliest natural disaster in the island’s recent history.
In late March this year a strong earthquake in the same vicinity as Sunday’s tremor killed one person and injured 86 others.
Taiwan lies near the junction of two tectonic plates and is regularly hit by earthquakes.