Full Text Search and Noise Words

  • I was under the impression that SQL2K handled noise words in the CONTAINS function, but I still get the error "A clause of the query contained only ignored words".

    What is the correct way to handle this - should the client app parse out any noise words or should it be done in the SP? How?

    Thanks for any help.

  • We actually get that here on the site sometimes, our search package uses FTS somewhere or other. Say "ho" is a noise word and you submit "ho ho ho", you wind up with an empty string. I think parsing might make sense, or just give the user the error message (or a better one).

    Andy

    http://www.sqlservercentral.com/columnists/awarren/

  • The annoying thing is that FTS throws out the complete search string even if only 1 word is a noise word. This example is in BOL:

    CONTAINS(pr_info, 'between OR king').

    Where are the noise words stored - can I access this from my SP in order to strip these words from my search string?

  • Noise words are stored in a text file.

    FROM BOL: SQL Server includes a standard list of noise words in the directory \Mssql\Ftdata\Sqlserver\Config

    If you are having troubles with the queries always being in the file you might want to create a table in your database with those words so that you can try to filter them out of the query before running it.

    Gary Johnson

    Microsoft Natural Language Group

    DBA, Sr. DB Engineer




    Gary Johnson
    Microsoft Natural Language Group
    DBA, Sr. DB Engineer

    This posting is provided "AS IS" with no warranties, and confers no rights. The opinions expressed in this post are my own and may not reflect that of my employer.

  • OK, I believe I have 3 options. Which of these would be the most efficient:

    1. A proc with a hard coded REPLACE for each of the noise words.

    2. A proc that reads a table containing the words and builds the REPLACE commands dynamically.

    3. A proc with a cursor to go through the keywords table, with only 1 REPLACE command.

    Thanks for any comments

Viewing 5 posts - 1 through 4 (of 4 total)

You must be logged in to reply to this topic. Login to reply