There was a time when SM didn't have the patience or knowledge to develop his Proxies. He lured the children away, but most of them ended up dead when he lost patience with them, with their innocence, inexperience, and weakness.
He learned, slowly, how to corrupt.
When we are young and helpless, He finds us and introduces some fatal flaw into our character. I am as flawed as anyone. Don't mistake the fact that I escaped as a sign that I wasn't as broken as the rest of you.
Then He makes us forget until he comes again.