Question about the Commonwealth - Any Help?

I was debating the Commonwealth Today - I said that the UK should not have gave India, South Africa, Egypt and all the former Commonwealth Countries back their Independence. I was then called a racist and a fascist - Am I being power hungry? DO you agree with me??