Abolishing caste requires destroying the sacred authority of shastras

Evidence

The destruction of caste is a reform which falls under the third category. To ask people to give up caste is to ask them to go contrary to their fundamental religious notions. It is obvious that the first and second species of reform are easy. But the third is a stupendous task, well-nigh impossible. The Hindus hold to the sacredness of the social order. Caste has a divine basis. You must therefore destroy the sacredness and divinity with which caste has become invested. In the last analysis, this means you must destroy the authority of the shastras and the Vedas. — Ambedkar, B R, Annihilation of Caste_ The Annotated…, 957