Yep, I often found that. I wanted to start speaking in a foreign language and because their English is better than my foreign language, they tend to get annoyed and start speaking English. Still, sometimes you can find someone willing to speak to you. Good luck with it :)