Finding groups of clustered data points

Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

All Forums

SQL Server 2012 Forums

Transact-SQL (2012)

Finding groups of clustered data points

Author

Topic

mattt
Posting Yak Master

194 Posts

Posted - 2013-05-21 : 09:31:38

Hi,

I have a table which records events:


create table #events
(  intRowId int identity(1,1),
   intItemId int,
   intUserId int,
   datEvent datetime)

It's a big table with many millions of rows, recording events against several thousand items and tens of thousands of users.

There's a select group of ten itemIDs I want to look for, but only when they occur in a certain pattern: I'm trying to find rows where all ten of these items have events registered against them for the same userID and close together in time, say 5 minutes.

I have absolutely NO IDEA how to go about this. One would assume partitioning is involved somewhere, but help, even just somewhere to get started, would be much appreciated.

Cheers,
Matt

James K
Master Smack Fu Yak Hacker

3873 Posts

Posted - 2013-05-21 : 10:18:36

Something like this?

SELECT
	*,
	EventCount
FROM
	#Events e1
	CROSS APPLY
	(
		SELECT COUNT(*) EventCount
		FROM #Events e2
		WHERE e2.intUserId = e1.intUserId
		AND ABS(DATEDIFF(mi, e1.datEvent, e2.datEvent)) < 5
		HAVING COUNT(*) > 10
	) e2

If that is logically what you need, and if it turns out the query is very slow, it can be optimized.

mattt
Posting Yak Master

194 Posts

Posted - 2013-05-21 : 10:33:39

Basically, yes, except I'm looking for ten specific items, rather than ten sequential events. But I guess that just means substituting an "in" clause for the "having"?

James K
Master Smack Fu Yak Hacker

3873 Posts

Posted - 2013-05-21 : 10:41:32

I think that may be right. Are you able to try the query on a small sample data set to see if it gives you the correct results?

mattt
Posting Yak Master

194 Posts

Posted - 2013-05-21 : 10:44:23

Yes. It seems to work best if I do it stepwise - so create a temp table with just the rows corresponding to the items in my list, and then use your query as is to find users which fit the pattern. Excellent, thank you.

Subscribe to SQLTeam.com

SQLTeam.com Articles via RSS

SQLTeam.com Weblog via RSS

- Advertisement -

Resources