May 21, 2007 at 4:40 pm
okay so I'm trying to de-dupe a list of customers from a table that has 165K records. Simple setup I have a query that defines 3 columns that I'm using and have the defaults for the fuzzy grouping set.
Anyone ever try to process this many records? I've been waiting for a couple hours now and it appears to not be processing anything...
Thanks,
Ben
Ben Sullins
bensullins.com
Beer is my primary key...
May 21, 2007 at 4:59 pm
Can we see the query you are using?
May 21, 2007 at 5:01 pm
This may be the issue...I'm concatenating a bunch of fields to come up w/ what would be considered a dupe...
So basically if the customer + full address + Comment is the same it's a dupe:
select
Customer + ShipAddr1 + ShipAddr2 + ShipAddr3 + ShipAddr4 + ShipAddr5 + ShipPostalCode + Comment as [rowtext]
from
myTable
Ben Sullins
bensullins.com
Beer is my primary key...
May 21, 2007 at 5:18 pm
No reason for this to take more than a few minutes even on a slow machine. ARe you sure you are not running a more complexe statement than this?
May 21, 2007 at 5:20 pm
That is the query in my 'Data Reader Source' that is being passed into my 'Fuzzy Grouping' then is to be inserted into a table on the same server...
Ben Sullins
bensullins.com
Beer is my primary key...
May 21, 2007 at 5:23 pm
Have you looked for locks of even dead locks?
May 22, 2007 at 11:54 am
It seems to be hung on the 'Finding Similar Records' step...i'll let it run for a while and see how it goes...
Ben Sullins
bensullins.com
Beer is my primary key...
May 22, 2007 at 11:59 am
Nice...so at 21% of the Finding Similar Records I get an 'Unhandled exception' that indicates corrupt memory. Any ideas???
Ben Sullins
bensullins.com
Beer is my primary key...
May 23, 2007 at 4:57 pm
So the answer is...
1.) I'm trying to run this locally and getting an error related to the amount of memory
2.) When deployed to the server I get an error: Product level is insufficient for component "Fuzzy Grouping:"
This is because we have standard installed which does not support Fuzzy functions. I thought Fuzzy Wuzzy was a bear! Not a product level!
lame :-/
Ben Sullins
bensullins.com
Beer is my primary key...
Viewing 9 posts - 1 through 8 (of 8 total)
You must be logged in to reply to this topic. Login to reply