Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2005 Forums
 Transact-SQL (2005)
 how to write this type of query

Author  Topic 

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2008-10-01 : 10:01:29
Hi,

I have a query I have to write, involving about 8 tables or so, and I believe it to be quite complicated. I'm not going to post all the details yet, as I am not looking for an exact answer, but just a general direction so I can attempt it in more detail first.

Basically I want to SELECT from a handful of tables, events that have happened, and order them DESC by date, and return this data back in the query.

The thing is all tables have different structures, but need to be sorted by the common field DATE.

This ultimately is a social networking feature that is going to let users know when people in their network have "added photos", or "updated their details".

How can I bring back 1 result set that can organize all this info?

Is 1 resultset the right approach? I don't see any other way ? I'm guessing if I do 1 resultSet, I would have to union all the columns somehow, and would end up having alot of NULL columns, and would create an extra column called TYPEID, that would allow me to process the row appropriately in the front end web application ?

The other option of course is to create a logging table of updates.


Any insight is greatly appreciated!

Thanks again,
mike123

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-10-01 : 10:16:20
You can start with two small simple tables demonstrating your need and businees rules.

Do you really need all columns from all tables?


E 12°55'05.63"
N 56°04'39.26"
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2008-10-01 : 10:34:14
quote:
Originally posted by Peso

You can start with two small simple tables demonstrating your need and businees rules.

Do you really need all columns from all tables?


E 12°55'05.63"
N 56°04'39.26"




Hey Peso,

Sure that would make sense. I don't know why it didnt come to mind, I will start with a small sample. And no, I don't need every single column from every table, but I do need the majority I believe. I'll post a sample in just a moment.

Thanks!
Mike123

Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-10-01 : 10:39:56
One thought is that build a string from each and one table and record, type the info and union all next table.



E 12°55'05.63"
N 56°04'39.26"
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2008-10-01 : 10:55:42
Please look at the details below.

As I see it, we will pass the SPROC 1 paramater called @userID

with @userID we will select all the users friends from the friends table. Once we have that list of "friendID"'s we will then query other tables to look for relavent rows. A relavent row is if the friendID is equal to the row "userID"

From the below tables we should have a query that returns data sorted by Date, the following info

When user "A"'s friends have added new photos
When users "A"'s friends have added new friends
and potentially
When users "A"'s friends have been added by new friends

Note that the friends relationship can be one way.

Of course after these 2 tables, there are half a dozen more to add or so.

I don't expect this to be very simple, and I'm hoping that executing it won't bog my server down like crazy. I guess we'll have to test and find out.

Your input is HUGELY appreciated. I wouldn't be going anywhere without it


Thanks very much!
mike123







--This table holds the users, its purpose in this post is so that we can JOIN onto it to get the users "nameOnline"


CREATE TABLE [dbo].[Users](
[UserID] [int] IDENTITY(1,1) NOT NULL,
[NameOnline] [varchar](15) NULL,
[Password] [varchar](20) NOT NULL,
[EmailAddress] [varchar](50) NULL,
[LastUpdated] [smalldatetime] NULL,
[GUID] [uniqueidentifier] NULL
)


--This table has 2 purposes. We will pull updates from this table, as well as establish which updates are relavent for the user. When we pass the query @userID, we will select all the users friends, and get updates on those users.

CREATE TABLE [dbo].[Friends](
[UserID] [int] NOT NULL,
[FriendID] [int] NOT NULL,
[Sort] [int] NOT NULL,
[dateAdded] [smalldatetime] NULL
)


--this table we will pull results from. with the @userID passed to the SPROC, we will determine all of the users friends who have added new photos, and bring back the appropriate columns. We need all columns except STATUS, and we need to join userID onto the USERS table, so we can bring back "nameOnline"


CREATE TABLE [dbo].[ExtraPhotos](
[counterID] [int] IDENTITY(1,1) NOT NULL,
[photoID] [tinyint] NOT NULL,
[userID] [int] NOT NULL,
[photoDate] [smalldatetime] NOT NULL,
[caption] [varchar](50) NULL,
[status] [tinyint] NOT NULL
)

Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-10-01 : 11:08:37
What about something similar to this?
CREATE PROCEDURE dbo.uspGetUserActivityList
(
@userID INT
)
AS

SET NOCOUNT ON

DECLARE @Friends TABLE
(
friendID INT PRIMARY KEY CLUSTERED,
nameOnline VARCHAR(15)
)

INSERT @Friends
(
friendID,
nameOnline
)
SELECT f.friendID,
u.nameOnline
FROM Friends AS f
INNER JOIN Users AS u ON u.userID = f.friendID
WHERE f.userID = @UserID

-- Present the resultset
SELECT f.nameOnline + ' added a new photo ' + CONVERT(CHAR(19), ep.maxDate, 120) AS msg,
ep.MaxDate AS theDate,
1 AS theType
FROM @Friends AS f
INNER JOIN (
SELECT userID,
MAX(photoDate) AS maxDate
FROM ExtraPhotos
GROUP BY userID
) AS ep ON ep.userID = f.friendID

UNION ALL

SELECT f.nameOnline + ' added a new friend ' + CONVERT(CHAR(19), u.maxDate, 120) AS msg,
u.MaxDate AS theDate,
2 AS theType
FROM @Friends AS f
INNER JOIN (
SELECT userID,
MAX(dateAdded) AS maxDate
FROM Friends
GROUP BY userID
) AS u ON u.userID = f.friendID

ORDER BY theType,
theDate DESC



E 12°55'05.63"
N 56°04'39.26"
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-10-01 : 11:29:58
And if you have a login history table in place, just pick the last time the @userID was logged in, and use that datetime information to only select records later than that.
CREATE PROCEDURE dbo.uspGetUserActivityList
(
@userID INT
)
AS

SET NOCOUNT ON

DECLARE @lastLogin DATETIME

SELECT @lastLogin = MAX(loginDate)
FROM LoginHistory
WHERE userID = @userID

DECLARE @Friends TABLE
(
friendID INT PRIMARY KEY CLUSTERED,
nameOnline VARCHAR(15)
)

INSERT @Friends
(
friendID,
nameOnline
)
SELECT f.friendID,
u.nameOnline
FROM Friends AS f
INNER JOIN Users AS u ON u.userID = f.friendID
WHERE f.userID = @UserID

-- Present the resultset
SELECT f.nameOnline + ' added a new photo ' + CONVERT(CHAR(19), ep.maxDate, 120) AS msg,
ep.MaxDate AS theDate,
1 AS theType
FROM @Friends AS f
INNER JOIN (
SELECT userID,
MAX(photoDate) AS maxDate
FROM ExtraPhotos
WHERE photoDate >= @lastLogin
GROUP BY userID
) AS ep ON ep.userID = f.friendID

UNION ALL

SELECT f.nameOnline + ' added a new friend ' + CONVERT(CHAR(19), u.maxDate, 120) AS msg,
u.MaxDate AS theDate,
2 AS theType
FROM @Friends AS f
INNER JOIN (
SELECT userID,
MAX(dateAdded) AS maxDate
FROM Friends
WHERE dateAdded >= @lastLogin
GROUP BY userID
) AS u ON u.userID = f.friendID

ORDER BY theType,
theDate DESC



E 12°55'05.63"
N 56°04'39.26"
Go to Top of Page

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2008-10-01 : 11:35:05
Just a doubt, shouldnt the first where condition be u.userID=@UserID rather than f.userID?
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-10-01 : 11:37:41
No, you want all friends of @userID, and the friend's name.



E 12°55'05.63"
N 56°04'39.26"
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2008-10-01 : 11:43:28
Hi Peso,

Currently just getting this error.

Msg 8120, Level 16, State 1, Procedure uspGetUserActivityList, Line 27
Column 'Friends.UserID' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.


any input very help
thanks again,
mike123
Go to Top of Page

visakh16
Very Important crosS Applying yaK Herder

52326 Posts

Posted - 2008-10-01 : 11:46:58
quote:
Originally posted by mike123

Hi Peso,

Currently just getting this error.

Msg 8120, Level 16, State 1, Procedure uspGetUserActivityList, Line 27
Column 'Friends.UserID' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.


any input very help
thanks again,
mike123



Sorry i'm not getting that error in Peso's posted query. ARe you sure you're using same query?
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2008-10-01 : 11:48:45
quote:
Originally posted by visakh16

quote:
Originally posted by mike123

Hi Peso,

Currently just getting this error.

Msg 8120, Level 16, State 1, Procedure uspGetUserActivityList, Line 27
Column 'Friends.UserID' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.


any input very help
thanks again,
mike123



Sorry i'm not getting that error in Peso's posted query. ARe you sure you're using same query?



thought I was.. let me double check :)

thx !
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-10-01 : 11:54:08
No worries. In my first version I missed to write GROUP BY.
That was quickly fixed but you managed to copy the code before the GROUP BY fix.



E 12°55'05.63"
N 56°04'39.26"
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-10-01 : 11:56:10
Maybe uspGetActivityListOfFriends is a better name for the stored procedure than uspGetUserActivityList?


E 12°55'05.63"
N 56°04'39.26"
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2008-10-01 : 12:00:32
Update: Not sure how I produced the error, but I recopied and the query runs great so far. What I am concerned about is adding more complicated issues like adding photos, and then not only showing the photo, but formatting the photoURL with HTML. I guess this ultimately could also be done in SQL too, but would be much easier, for me , in the front end.

What do you think ?


The other thing I am deliberating over, is perhaps its best to have some type of logging table that I could query from directly ? This would only be done for speed reasons. So far this query is running really fast, but its quite simple still. This way would definately be better if it can run fast enough, so perhaps its best to try with some more tables :)


Thoughts?

Thanks once again,
mike123
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2008-10-01 : 12:01:48
quote:
Originally posted by Peso

Maybe uspGetActivityListOfFriends is a better name for the stored procedure than uspGetUserActivityList?


E 12°55'05.63"
N 56°04'39.26"




yes definately :)
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-10-02 : 08:46:41
Try on!

Add one more table to the output until you either reach your goal or the performance is really bad.



E 12°55'05.63"
N 56°04'39.26"
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2008-10-02 : 09:22:24
Hey Peso,

after I've been able to think about this for today, I think this query is great, and it very well could be my best option, but I still have some concerns I would like to ask about

As we add more complicated tables, theres some advanced text string manipulation that is really best suited to be done in the web application.... (for example bringing back an image, we need to add html to it etc, line breaks between certain rows, formatting photos together if they are updated next to each other)


To compare this method, with the method that I think would make my front end application function easiest.

If the data looked like this, do you think this is wrong? slow? inefficient ? Ultimately it could be 20 columns wide or so.

Any thoughts much appreciated.

Thanks!
mike123



UserID FriendID dateAdded counterID,userID,caption,photoDate
100 2000 2008-01-01 NULL NULL NULL NULL
100 3000 2008-01-01 NULL NULL NULL NULL
100 4000 2008-01-01 NULL NULL NULL NULL
100 5000 2008-01-01 NULL NULL NULL NULL
NULL NULL NULL 500 499 them 2007-01-01
NULL NULL NULL 501 499 me 2007-01-01
NULL NULL NULL 502 499 u 2007-01-01
NULL NULL NULL 503 499 us 2007-01-01

Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-10-02 : 09:27:37
I think you should query the onlinename too, right?

You can do your other idea to, have a log table and a trigger on every of the 20 tables or so inserting into log table with the log text you wish. it will somewhat slow down each and one update/insert, but the presentation will be fast.



E 12°55'05.63"
N 56°04'39.26"
Go to Top of Page

mike123
Master Smack Fu Yak Hacker

1462 Posts

Posted - 2008-10-02 : 09:42:31
quote:
Originally posted by Peso

I think you should query the onlinename too, right?

You can do your other idea to, have a log table and a trigger on every of the 20 tables or so inserting into log table with the log text you wish. it will somewhat slow down each and one update/insert, but the presentation will be fast.



E 12°55'05.63"
N 56°04'39.26"




Hi Peso,

Yes you are definately correct, I would need the user_nameOnline and the friend_nameOnline.

And in the case of photos I would have to do a join too with userID to get the photo_nameonline

Just posted like that so you could see the structure, with so many extra column names and NULLs.

I've never actually used a trigger so I will read into them, I can see a logging table being quite fast... But perhaps we can get good speed without it. If I did a logging table, I assume this format with all the extra columns would be a decent approach ?


Thanks once again,
Mike123

Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2008-10-02 : 10:00:53
No. The logging table is best queried of all text already is there.

Logging table would only need four columns, where dtAction and userid is part of clustered index
dtAction | idAction | userID | logText

The overhead for each and one trigger to build the string is almost negliable.
It is in the area of a few milliseconds extra per update and insert, per table.

If you can live with an insert taking a few milliseconds extra, the query for log text will be really fast, beacuse all JOINS to produce the text is already done.

If you have a log table like the one suggested, you only need to make one join against friends table

SELECT lh.logText
FROM LogHistory AS lh
INNER JOIN Friends AS f ON f.friendsID = lh.userID
WHERE f.userID = @userID
AND lh.dtAction >= @SomeDateTime
ORDER BY lh.idAction





E 12°55'05.63"
N 56°04'39.26"
Go to Top of Page
    Next Page

- Advertisement -