Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 General SQL Server Forums
 New to SQL Server Programming
 Detecting Duplicates

Author  Topic 

ConradK
Posting Yak Master

140 Posts

Posted - 2010-06-11 : 15:01:58
So I've got a table that is absolutely huge, and running aggregate queries is bogging it down and leading to almost nothing. I just want to know whether there are any duplicates within a certain column of that table. I'll decide what to do from there, but first I need just a way to KNOW whether or not there are duplicates within the able.

keep in mind that

select

count(DescriptionFields.Description)
from DescriptionFields
group by Description


seemed to absolutely tank the server, so... think efficiency here. I am out of ideas.

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2010-06-11 : 15:06:50
select Description, count(*)
from DescriptionFields
group by Description
having count(*) > 1

How many rows are in this table?

Tara Kizer
Microsoft MVP for Windows Server System - SQL Server
http://weblogs.sqlteam.com/tarad/

Subscribe to my blog
Go to Top of Page

ConradK
Posting Yak Master

140 Posts

Posted - 2010-06-11 : 15:08:19
3 quarter of a million
Go to Top of Page

ConradK
Posting Yak Master

140 Posts

Posted - 2010-06-11 : 15:10:04
and each 'description' is a full flegged website page.

I don't want to pull just the unique ones, I want to know if each entry is unique or not. It may be, it may not be. I'm not familar with this database.
Go to Top of Page

ConradK
Posting Yak Master

140 Posts

Posted - 2010-06-11 : 15:11:52
each entry has its own unique indexid, but what I want to know is whether or not each entry is a unique webpage, or if it contains duplicates. Neither reality would surprise me.
Go to Top of Page

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2010-06-11 : 15:17:00
That is a *small* table these days, but I suppose the size of that column would make the table bigger.

What you are trying to do needs to scan the entire table, so it can't be fast.

Tara Kizer
Microsoft MVP for Windows Server System - SQL Server
http://weblogs.sqlteam.com/tarad/

Subscribe to my blog
Go to Top of Page

ConradK
Posting Yak Master

140 Posts

Posted - 2010-06-11 : 15:21:07
I was afraid of this. I keep hoping the road blocks I run into are a part of my novice naivety but are turning out to just be the physical constraints of the hardware more and more....

Thanks for your help.
Go to Top of Page

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2010-06-11 : 15:21:46
You're welcome.

Tara Kizer
Microsoft MVP for Windows Server System - SQL Server
http://weblogs.sqlteam.com/tarad/

Subscribe to my blog
Go to Top of Page
   

- Advertisement -