October 22, 2009 at 7:11 am
The app is custom tailored for our needs. It's an ocr that scans a pile of 100s of pages. It detects what the page is and then it extracts information and saves it in the DB in a element-value table. It also saves the full pdf in another table.
So you basically end up with 2 huge tables in the DB. Our original problem was that the search software which they provide was slow... since each search does a scan of a 1.8 GB table, there's not much more we can do in there.
October 22, 2009 at 7:14 am
Andre Gous (10/21/2009)
Founder, CEO, Chief GeekPrecision Quality Software, Inc.
Just curious, is Chief Geek an actual title? 'Cause if so, I need to apply for Craven Underling
---------------------------------------------------------
How best to post your question[/url]
How to post performance problems[/url]
Tally Table:What it is and how it replaces a loop[/url]
"stewsterl 80804 (10/16/2009)I guess when you stop and try to understand the solution provided you not only learn, but save yourself some headaches when you need to make any slight changes."
October 22, 2009 at 7:35 am
Just my 2ct...
In my prereq sheet, our window installers are using, I always request the disks to be formatted using the largest possible block size available.
I personally don't think SQLExpress should be used to actually store the LOB data (pdf files in your case).
just because the size of these lobs will fairly soon become an issue.
I've built a little app (access frontend / sqlexpress db) in which I store 4KB pictures externally on a documented location.
The backup procedure uses that location to robocopy the data to the safezone, together with the database backup.
The app just reads the picture location and ID and fetches it directly, not using the db engine for that, so keeping the sqlserver memory footprint as small as possible.
Off course not everybody has write access to that picture folder.
Johan
Learn to play, play to learn !
Dont drive faster than your guardian angel can fly ...
but keeping both feet on the ground wont get you anywhere :w00t:
- How to post Performance Problems
- How to post data/code to get the best help[/url]
- How to prevent a sore throat after hours of presenting ppt
press F1 for solution, press shift+F1 for urgent solution 😀
Need a bit of Powershell? How about this
Who am I ? Sometimes this is me but most of the time this is me
October 22, 2009 at 7:46 am
I know Al.... VENDOR APP is the keyword in my question :w00t:.
Anyhow if you're going to waste time documenting every report in our compagny and customize the OCR for them, why not just had a freaking table and index them instead of unnormalizing the data in the first place and then making it impossible to fetch quickly?.. Just because they didn't know better and now it's too late!
October 22, 2009 at 2:36 pm
Dear GSquared,
Thank you for the guidance - both political and technical. What you wrote makes perfect sense to me.
Regards,
Regards,
Andre Gous
Founder, CEO, Chief Geek
Precision Quality Software, Inc.
Divisions include:
FallonWebSites.com
NevadaPaperless.com
October 22, 2009 at 2:48 pm
Ninja,
Thanks for the explanation. Your vendor's approach certainly seems reasonable on the surface, but I can understand why ultimately problems occurred. The concept of doing a full-text search is messy but there are licensed technologies that one can integrate so that the text search is blazingly fast, 1Gb per second according to one claim I've heard recently. of course, that presumes a great many premises without which the number is meaningless.
I'm trying to maintain a fine line between being sympathetic, helpful, violating forum ettiquette and intruding on your existing relationship with your vendor.
But, if he's truly out of his depth and struggling, he might actually appreciate (as opposed to resenting) some guidance.
Please let me know if you would like more information. I feel a little awkward withholding what I know when it seems to solve the problem your vendor is having.
At some point, this forum might not be the most appropriate venue either, so please feel free to contact me offline (again, if it's appropriate to the rules of this forum.
Regards,
Andre
Regards,
Andre Gous
Founder, CEO, Chief Geek
Precision Quality Software, Inc.
Divisions include:
FallonWebSites.com
NevadaPaperless.com
October 22, 2009 at 2:59 pm
I appreciate the humor in your question. "Chief geek" would puzzle folks who receive my physical business cards, but I come close to saying it explicitly anyway. I unveil my true nature explicitly only in places like this forum, where like-minded folks are.
Since you asked: seriously, it's nice to be a geek personablly and to have created a company that's totally focused on keeping developers happy. The shock absorbers I've had to put in place between what developers like and the harsh realities of the market are small. Everyone works flexible hours, takes off whenever they want, etc. but even so clients end up getting good software on time, and having a very high level of support and responsiveness (even though it might happen while Lance is driving to Utah or Jamie is taking a break from surfing a set of waves on Hanalei bay). It's nice to be focused on that -- unapologetically. There are wine coolers and beers in the company 'fridge and yet everyone is professional as to their work. Trusting people -- what a concept. An acquaintance of mine has tried to set up a programmer's shop that he rules with an iron fist and yet he's puzzled why his developers keep being burned out and leaving. Go figure.
Regards,
Andre Gous
Founder, CEO, Chief Geek
Precision Quality Software, Inc.
Divisions include:
FallonWebSites.com
NevadaPaperless.com
October 22, 2009 at 8:47 pm
WAY OFF TOPIC THEN I'LL SHUT UP
I'm glad to hear others talk about Timex Sinclear's and Comadore's. Now I don't feel so old and alone thinking about my Northstar 80 and binary & assembler. Anyone for some warm milk and cookies?
October 23, 2009 at 6:22 am
Tim Riley-487603 (10/22/2009)
WAY OFF TOPIC THEN I'LL SHUT UPAnyone for some warm milk and cookies?
Sure, I will join you just as soon as my nurse changes my depends and gives me a sponge bath:-D
"There are three kinds of men. The one that learns by reading. The few who learn by observation. The rest of them have to pee on the electric fence for themselves." -- Will Rogers
Viewing 9 posts - 31 through 38 (of 38 total)
You must be logged in to reply to this topic. Login to reply