After addressing delegates at the opening of the Johor Darul Ta'zim football club congress on Wednesday, FAM president Tunku Ismail Ibrahim told reporters that there is a conspiracy by some FAM exco members to remove him as the president. But he said he would not step down easily.