In addition to Max's great answer about removing filler words, there are several areas that one can improve their public speaking abilities. The first area is in the actual content of the speech, and the second area is through delivery.
Through content, the best tip is to know your audience! Pick a topic that interests and relates to the audience and choose analogies and examples that your audience can understanding. Your vocabulary should not be above or below the audience's level. Brainstorm what your audience members have in common. What are some possible topics that those audience members would want to hear about? For example, if I'm speaking to a PTA group full of mothers with children, what topics would be most interesting to them? In contrast, if I'm speaking in front of a group of faculty members at a college, what topics would be most interesting to them? There should be obvious differences in the topics, tone, and vocabulary levels between the two different groups. However, keep in mind that the topic should be interesting to you as well. If you choose a topic that is not interesting to you, your speech will tend to lack passion and energy. So choose a topic that you have in common with your audience members.
Delivery of a speech has lots of components. Eye contact is a good suggestion, but make sure to distribute your eye contact evenly throughout the room or group. You should spend about 3-4 seconds on each person that you look at. It's enough to try to read that person and see if they are paying attention and are finding your speech interesting but not long enough to be creepy/awkward.
When looking at your audience members, don't panic if they seem uninterested! Keep in mind that audience members undergo lots of internal and external noise. Internal noise such as "I'm starving" or "I wish this headache would go away" or "I'm worried about that argument I had with my friend this morning". External noise could be "Good grief, this room is freezing" or "What is that weird clicking noise I keep hearing?" Be understanding of your audience members; don't stress over keeping the attention of every audience member. Plow on with your speech and keep it cool.