Is there a recommended data compression utility?

  • We’re searching for a product.

    We are currently storing XML text “blobs” in our database, in TEXT columns. This requires N storage space, where N is far, far too many gigabytes. I have done some testing, and discovered that if the XML (as individual files) is run through Winzip, the resulting binary would require .05N space, resulting in far, far fewer gigabytes. (If hard drive space is cheap, then it’s Ok to take it out of your salary, right?)

    Next bit: One server, N clients. Which server (processor) is going to perform the compression/decompression, the one database server that everyone feverishly accesses, or the myriad client application servers waiting for their data with all those spare CPU cycles? On to the next bit.

    The data is used by applications. (It’s merely stored in SQL Server). What we’d want is a COM, API, or otherwise programmatic interface to the compression/decompression utility. Our developers have stated that Winzip comes with no such interface, meaning we can’t install and use it.

    The question: is there a utility out there that anyone knows about or can recommend that can do this? That we can purchase/license for a reasonable fee and include in our application suite that allows programmatic (as opposed to UI-accessed) compression and decompression functionality?

    (Doing any of the “convert XML to conventional table structures” routines won’t work, because (a) we don’t need to do it, and (b) you really don’t want to see our XML…)

    Philip

  • It seems like sql server ought to be able to this by itself since as part of replication process you can have it "compress" snapshot files. Of course BOL doesn't have anything on it other than that. No doubt some undocumented stored procedure. Besides, maybe you shouldn't clog up a database engine's memory and processes with doing compression.

    You could run xp_cmdshell with a separate commandline zip utility. You could even make a different server do the actual compressing so as not to effect SQL Server's performance.

    Winzip does have a beta commandline utility.

    http://www.winzip.com/wzclineb.cgi

    PKZip also has a commandline feature:

    http://www.pkzipstore.com/dr/v2/ec_MAIN.Entry10?V1=490231&PN=1&SP=10023&xid=39892&DSP=&CUR=840&PGRP=0&CACHE_ID=0

    Of course, there are also about a Kagillion zip programs out there and no doubt many of them will use a commandline interface.


    "I met Larry Niven at ConClave 27...AND I fixed his computer. How cool is that?"
    (Memoirs of a geek)

  • http://www.xceedsoft.com/products/ZipNet/

    http://www.chilkatsoft.com/dotNetZip.asp

    http://www.example-code.com/vb/default.asp

    FREE!

    http://www.organicbit.com/zip/

    http://www.gzip.org/zlib/

    http://www.winimage.com/zLibDll/

    -------

    at us, very deafly, a most stares

    collosal hoax of clocks and calendars

    eecummings


    -------
    at us, very deafly, a most stares
    collosal hoax of clocks and calendars

    eecummings

Viewing 3 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic. Login to reply