According to Sindh chief minister Murad Ali Shah, most of the underpasses were flooded and there was nowhere to pump the floodwaters.
He said an unprecedented 126 millimeters (4.96 inches) of rain fell in three hours.
The streets of affluent neighborhoods, such as the Department of Defense Housing Authority complex and Clifton, were flooded, with water gushing into homes.
The main streets of Karachi, which are home to financial institutions and bank headquarters, including the central bank of Pakistan, were flooded, and rescue services used boats to reach stranded people.
The Pakistan Navy also took part in the rescue operations.
“Deeply saddened by the tragic losses due to heavy rains in Karachi,” Prime Minister Shahbaz Sharif wrote on Twitter. He offered all possible support to the Karachi administration.
The highway connecting the city of Quetta with Karachi was also closed to traffic as sections of it were washed away by flash flooding.
In the southwestern province of Balochistan, about a dozen villages in the Lasbela district were flooded when the Winder River burst its banks and flooded homes.
According to government official Farhan Suleman Ranjo, the navy and local authorities have already rescued about 500 people and local authorities from the affected villages.
Last week, 64 people died in different parts of the Balochistan province as a result of a burst of eight dams due to flood waters.
The Met Office is predicting more rain in the coming days.