Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2008 Forums
 Transact-SQL (2008)
 Removing Duplicate Words

Author  Topic 

sigmas
Posting Yak Master

172 Posts

Posted - 2013-01-11 : 09:03:56
Hi,
A string sample value:
declare @s varchar(50) = 'dups dups dups value1 value1 value3 value1 value dups'

I want remove duplicate words in that string. so result must be:
'dups value1 value3 value'

Could you help me?

nigelrivett
Master Smack Fu Yak Hacker

3385 Posts

Posted - 2013-01-11 : 09:06:21
You can parse the string putting the words into a table then recreate the string using distinct values.

==========================================
Cursors are useful if you don't know sql.
SSIS can be used in a similar way.
Beer is not cold and it isn't fizzy.
Go to Top of Page

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2013-01-11 : 09:08:36
similar to

http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=182027

declare @s varchar(50) = 'dups dups dups value1 value1 value3 value1 value dups'
declare @sunique varchar(50)

SELECT DISTINCT @sunique= COALESCE(@sunique,'') + Val + ' '
FROM dbo.ParseValues(@s,' ') f

SELECT @sunique


------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Go to Top of Page

sigmas
Posting Yak Master

172 Posts

Posted - 2013-01-11 : 09:16:29
Thanks,
Splitting words is good. but is not another solution(optimized/intelligent) for this? I mean when even there are not duplicate words or just two duplicate word we need to split all of string.
Go to Top of Page

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2013-01-11 : 09:31:34
quote:
Originally posted by sigmas

Thanks,
Splitting words is good. but is not another solution(optimized/intelligent) for this? I mean when even there are not duplicate words or just two duplicate word we need to split all of string.


Unless you parse string you cant determine whether it has duplicates. so anyways there should be parsing logic.
Another way is this


declare @s varchar(50) = 'dups dups dups value1 value1 value3 value1 value dups'
declare @sunique varchar(50)
SELECT @sunique = COALESCE(@sunique,'') + v
FROM
(
SELECT DISTINCT m.n.value('.','varchar(100)') + ' ' AS v
FROM (SELECT CAST('<Values><Value>' + REPLACE(@s,' ','</Value><Value>') + '</Value></Values>' AS xml) AS u )t
CROSS APPLY u.nodes('/Values/Value')m(n)
)r

SELECT @sunique


------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Go to Top of Page

sigmas
Posting Yak Master

172 Posts

Posted - 2013-01-11 : 09:42:37
quote:
Originally posted by visakh16

quote:
Originally posted by sigmas

Thanks,
Splitting words is good. but is not another solution(optimized/intelligent) for this? I mean when even there are not duplicate words or just two duplicate word we need to split all of string.


Unless you parse string you cant determine whether it has duplicates. so anyways there should be parsing logic.
Another way is this


declare @s varchar(50) = 'dups dups dups value1 value1 value3 value1 value dups'
declare @sunique varchar(50)
SELECT @sunique = COALESCE(@sunique,'') + v
FROM
(
SELECT DISTINCT m.n.value('.','varchar(100)') + ' ' AS v
FROM (SELECT CAST('<Values><Value>' + REPLACE(@s,' ','</Value><Value>') + '</Value></Values>' AS xml) AS u )t
CROSS APPLY u.nodes('/Values/Value')m(n)
)r

SELECT @sunique


------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/




Could you explain this method?
Go to Top of Page

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2013-01-11 : 10:15:50
Its basically forming an xml document out of string of input data and then using T-sql XML functions to parse out data as table and applying distinct over it

------------------------------------------------------------------------------------------------------
SQL Server MVP
http://visakhm.blogspot.com/

Go to Top of Page

sigmas
Posting Yak Master

172 Posts

Posted - 2013-01-11 : 10:22:17
OK, if be some words same with '<values><value>' at the string then the query will return correct result or not?
Go to Top of Page

Jeff Moden
Aged Yak Warrior

652 Posts

Posted - 2013-01-12 : 20:40:34
quote:
Originally posted by sigmas

OK, if be some words same with '<values><value>' at the string then the query will return correct result or not?



Not.

Your turn. What would the source of such information with duplicated words be and why does it have the problem of producing duplicated words to begin with?

--Jeff Moden
RBAR is pronounced "ree-bar" and is a "Modenism" for "Row By Agonizing Row".

First step towards the paradigm shift of writing Set Based code:
"Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column."

When writing schedules, keep the following in mind:
"If you want it real bad, that's the way you'll likely get it."
Go to Top of Page
   

- Advertisement -