Toastmasters for ESL?

Someone recommended Toastmasters so I went to one club in my area. I was surprised that everyone there was native speakers who were already proficient in English and their problem was not English but way beyond that. They were also public speaking experts. I didn't understand 90% of what's been said in the meeting because of my English. Would it help me to attend such a thing where other members are way above my level both in English and in public speaking? I felt so intimidated and uncomfortable while I was there.