Compression tool???

Question

Compression tool???

DAVNovak

Hall of Fame

Points: 3120
More actions
April 21, 2005 at 8:57 am

#92938

Does anyone know of a compression tool that can be used in SQL Server that will work at the field level?

Dave

Viewing 8 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic. Login to reply

Wayne West SSC-Insane Points: 22586 More actions · Answer 1

I was hoping someone might have replied to you, it sounds interesting. What exactly are you trying to accomplish? Are you talking about compressing images or large text files for storage in large objects?

-----
[font="Arial"]Knowledge is of two kinds. We know a subject ourselves or we know where we can find information upon it. --Samuel Johnson[/font]

Steve Jones - SSC Editor SSC Guru Points: 736255 More actions · Answer 2

Is this data compression inside the database? You could probably use any type of zip tool to compress data before you put it in, but it would make searching impossible.

The other thing is that using varchar v char is a form of compression. You're only using the space that you need.

DAVNovak Hall of Fame Points: 3120 More actions · Answer 3

Wayne,

Yes, that is exactly what we would like to do. We are looking at building a Bill Statement presentation web app. We would need to store 6 months of bill statements for 1.2 million customers and growing. We are looking at storing the actual individual bill statements as either xml, txt, pdf, or tif files. If it is possible, we would like to compress these files and store them in the DB instead of as individual files on the network. I was hoping either Frank Kalis, or Andy and/or Steve would reply if they had seen or heard of such a tool.

Dave

Wayne West SSC-Insane Points: 22586 More actions · Answer 4

Well, I had a very bad suggestion, but since this is going to be a web app it is utterly non-viable. Trust me, it was ugly and would have subjected me to much public ridicule on this forum. But I will tell you about it if you really want to know....

ANYWAY, I would approach it through using your middle tier to compress/decompress the records using a PKWARE compression API (http://www.pkware.com/products/developer/). I've never done any programming like this, so I have absolutely zero suggestions on how this would be implemented, but I think this would be a decent solution. I don't think storing them as XML would be an improvement, see Don's article at http://www.sqlservercentral.com/columnists/dpeterson/isxmltheanswer.asp. I would compress the data through middleware, then generate a PDF through reporting software so you wouldn't have any cross-platform/cross-browser issues.

I personally have absolute faith in PKWARE's compression algorithms, their software is lossless and I've never had a problem with it. Your mileage may vary, naturally.

FWIW. Good luck!

-----
[font="Arial"]Knowledge is of two kinds. We know a subject ourselves or we know where we can find information upon it. --Samuel Johnson[/font]

yolousa Old Hand Points: 367 More actions · Answer 5

There is a tool (SQLCompress.NET by Lago Information Technologies http://www.sqlcompress.net) about to be released (is in RC2 as I type). It does server side column level compression, it is free and does not require any application modification. Only on caveat, you need SQL Server 2005.

Ross-379529 SSC Eights! Points: 878 More actions · Answer 6

I have to ask - what size are the uncompressed forms of the files you are planning on storing within the database - and what size are you expecting them to be compressed by? - as you are already storing these external to the DB with I assume index pointers to the location of the file. . Why or what has been the catalyst to look into storing these within the database - The reason I ask is that I went the opposite direction (removed my blobs from the database and stored externally ) I was dealing with files - between .5MB - 30MB compressed (2.1:1 lossless) - the reason I moved them external so I could multithread my reading capabilities (i.e. download to client multiple files similtaneously) and also separate the load balancing for the database as opposed to blob transactions to separate servers for performance and bandwidth considerations.

** What you see, Depends on what you Thought, Before, You looked! **

DAVNovak Hall of Fame Points: 3120 More actions · Answer 7

This project was killed a few months ago. Put I do thank you for your recent replies. Actually, it was my boss that wanted to find out if there was a way do file compression inside of SQL Server. My suggestion to him was to compress and save the file out side of the server and save the path in the DB. But, I do thank you for the replies. The product that Yoel mentioned looked interesting.

Dave