"I have taken a decision to step down as the deputy mayor and from being a councillor," he told reporters in Pietermaritzburg.
Shelembe, who held the position for less than a month, quit after a protest that brought the Pietermaritzburg CBD to a halt last Friday.
Protesters gave Shelembe an ultimatum to step down by the end of this month, or "face an ungovernable city".
Speaking on behalf of the protesters, ANC activist Nana Mnandi, said they were willing to bring the city to its knees and render it ungovernable unless their concerns about corruption were addressed.
ANC MPL Makhosi Khoza this week also called for Shelembe to be removed from his job.
His appointment as deputy mayor after the local government elections was met with fierce opposition, not only from opposition parties, but also from ANC members and Cosatu.