When it comes to determining the safety profile/ toxicity of a drug molecule, most standard computational models focus on predictive accuracy. However, when the patient's health is on the line, we cannot rely on a black box model. Our main aim through this project is to shift the focus on interpretability of the model using explainable AI and find out what kind of chemical substructures help determine the toxicity of an input molecule.