This script uses a recursive CTE to identify non-allowed characters from a column in a table. In this case, the PATINDEX is set up to consider alphanumerics as the only valid characters.
You may need to set the MAX RECURSION limit if you think there will be > 100 invalid characters in any particular string.
I have tested this script using a 100,000 row test harness on 100 character strings salted with about 20-25% non-alphanumeric characters and it ran in about 28-45 seconds. 1,000,000 rows ran in about 5-6 minutes. Variability was probably caused by random invalid characters being assigned to each string.
[Note to editor] Sorry but I couldn't figure out how to get my script into the field for it on this page.