Hi Alan,

thanks for reading them.  It would have been nice to see your longer answer.

Microsoft has(had?) a reseearch group on data cleaning https://www.microsoft.com/en-us/research/project/data-cleaning/
which resulted...

Here is another one:

https://www.researchgate.net/profile/Panos_Ipeirotis/publication/2922719_Approximate_String_Joins_in_a_Database_Almost_for_Free/links/0deec530b377b82785000000/Approximate-String-Joins-in-a-Database-Almost-for-Free.pdf

Lauri

Hi,

there has , in fact been some research on fast qgrams with SQL, see for instance

http://www.cs.columbia.edu/~gravano/Papers/2001/deb01.pdf

Lauri

Well, not really procedural, since all those "side tables" can be created using "vanilla" SQL. Your approach may run into problems if you are comparing thousands of values with each...

There has actually been some research on this approach,  see for instance:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.14.6750&rep=rep1&type=pdf

And Microsoft SSIS uses a similar approach in it's "fuzzy matching" component, see:
https://www.microsoft.com/en-us/research/project/data-cleaning/

Thank you for your kind words!

Indeed, there can be any number of reasons for overlapping indexes and it certainly would be irresponsible to just go and drop them...

May I suggest an alternative script for the same purpose? It takes into consideration ascending/descending but gives false positives for "mirror image" indexes, e.g.

(c1 , c2 desc, c3) can be...