I too have never had a problem with the slurry on the bottom of the flask after giving it some hard swirls with a little of the remaining beer above it. Just swirl or even shake it harder, it won't hurt the yeast. Just be sure to do it with a sanitized foil on top so you don't lose any to the outside.
"A bad man is a good man's job, while a good man is a bad man's teacher."