Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2008 Forums
 Transact-SQL (2008)
 High volume scoreboard querying (optimising issue)

Author  Topic 

PedroSQL
Starting Member

2 Posts

Posted - 2011-05-19 : 17:10:16
Dear sql gurus,

I have a system that collects game scores from each round & totals them in a PlayerTotals table.

The PlayerTotals table is an average 200,000 rows.

It is constantly being updated with new scores and players.

I need to query the table, for a scoreboard, which can be sorted by score, kills & deaths.

Querying the live table whilst it is also being updated is presumably bad.

I decided to create duplicate tables that have already been sorted (PlayerTotals_Score, PlayerTotals_Kills, PlayerTotals_Deaths).

I plan to run the updates every hour, then the scoreboard queries THESE tables instead of the 'live' table (paging, find position of specific player, etc)

Below is my initial attempt. Ultimately I'm trying to produce fast sorted/paged/filtered scores, and not affect update speed of the original source.

1) Am I correct that I should do this away from the 'high volume update' table PlayerTotals?
2) Should I be puling this to another database (on same server) to process?
3) Is there just a much better way of doing all this?

All advice or just random suggestions welcomed!

PS. I presume I should be locking these tables from the website whilst deleting and refilling?

[edit] Code below took 3 minutes to run the first time, and just hung the server the second time :(





DECLARE @Temp TABLE (
[UserId] [bigint] NOT NULL,
[CharacterName] [varchar](30) NOT NULL,
[Kills] [int] NOT NULL,
[Deaths] [int] NOT NULL,
[Score] [int] NOT NULL,
[LastSeen] [datetime] NOT NULL,
[TeamId] [int] NOT NULL,
[GamesPlayed] [int] NULL,
[Kit] [nchar](100) NULL,
[DayStartPosition] [int] NULL,
[KDR] [real] NULL)

insert into @Temp (UserId,CharacterName,Kills,Deaths,Score,LastSeen,TeamId,GamesPlayed,Kit,DayStartPosition,KDR) SELECT * FROM PlayerTotals

delete from PlayerTotals_Score

insert into PlayerTotals_Score (RowId,UserId,CharacterName,Kills,Deaths,Score,LastSeen,TeamId,GamesPlayed,Kit,DayStartPosition,KDR)
SELECT * FROM ( SELECT ROW_NUMBER() OVER (ORDER BY Score DESC) AS RowNum, *
FROM @Temp) AS RowConstrainedResult
ORDER BY RowNum

delete from PlayerTotals_Kills

insert into PlayerTotals_Kills (RowId,UserId,CharacterName,Kills,Deaths,Score,LastSeen,TeamId,GamesPlayed,Kit,DayStartPosition,KDR)
SELECT * FROM ( SELECT ROW_NUMBER() OVER (ORDER BY Kills DESC) AS RowNum, *
FROM @Temp) AS RowConstrainedResult
ORDER BY RowNum

delete from PlayerTotals_Deaths

insert into PlayerTotals_Deaths (RowId,UserId,CharacterName,Kills,Deaths,Score,LastSeen,TeamId,GamesPlayed,Kit,DayStartPosition,KDR)
SELECT * FROM ( SELECT ROW_NUMBER() OVER (ORDER BY Deaths DESC) AS RowNum, *
FROM @Temp) AS RowConstrainedResult
ORDER BY RowNum

robvolk
Most Valuable Yak

15732 Posts

Posted - 2011-05-19 : 18:16:41
quote:
Querying the live table whilst it is also being updated is presumably bad.
Why presume that? You're querying it to populate the table variable.

I'd suggest a few things (assuming you haven't tried them yet):

1. Create indexes on the Score, Kills, and Deaths columns on the PlayerTotals table and query it directly with the correct ORDER BY clause.
2. If #1 not possible, don't use a table variable. Insert the data into a temp table and create the indexes as described.
3. Use a single temp table instead of 3.
4. Limit the number of rows returned in the query, like TOP 100 or TOP 1000 at most. Use ROW_NUMBER() to navigate/return pages of data only if requested (i.e. not Page 1).
5. Consider NOLOCK and FAST query hints if the UPDATE query is being impacted.
Go to Top of Page

PedroSQL
Starting Member

2 Posts

Posted - 2011-05-19 : 19:14:24
Thank you for your suggestions robvolk.

I thought I should move to a table variable to prevent the rest of the queries from holding up the table longer than needed. Is that a false assumption? I've had a couple of painful server freezes whilst testing this stuff.

1) I will read up more on indexes.
2) I read table variables are quicker than temp tables?
3) These are not temp tables. They are 'cache' tables that the website uses, so the grunt work of ordering and adding rownum is done just once an hour. I figured running the type of query below would be even faster without the ordering. Most scoreboards I see online are updated periodically, presumably for these kind of query speed/lock-up reasons?
4) You're absolutely right about limiting to 1,000 rows, silly me. It now takes just 13 seconds to run.
5) I will read up more on NOLOCK and FAST

I just assumed there was a better way to do things rather than have a frequently visited website querying the same table that hundreds of servers are updating.

The main thing I want to avoid is any lock-ups, as I can only check in on things each evening after work, and don't want to find it's been hung all day.

My original 'direct' multi parameter, paged, filtered query is below, in case it helps anyone else.
It actually only takes 1 second to run on a 200,000 record table.


CREATE PROCEDURE [dbo].[spGetFilteredScoreboard]
@Start int = null,
@Side int = null,
@Kit varchar(30) = null,
@Name varchar(30) = null ,
@OrderBy varchar(30)
AS

declare @Max Integer
declare @Position Integer

if (@Name is not null)
begin
set @Position = (SELECT RowNum FROM ( SELECT ROW_NUMBER() OVER
(
ORDER BY
CASE
WHEN @OrderBy = 'Score' THEN Score
END DESC,
CASE
WHEN @OrderBy = 'Kills' THEN Kills
END DESC,
CASE
WHEN @OrderBy = 'Deaths' THEN Deaths
END DESC
) AS RowNum, * FROM HeroTotals WHERE
(@Side is null or TeamId=@Side) and
(@Kit is null or Kit=@Kit)) AS RowConstrainedResult WHERE CharacterName = @Name)
set @Start = @Position - 50
end

if (@Start < 1)
set @Start = 1

set @Max = @Start + 100;

SELECT * FROM ( SELECT ROW_NUMBER() OVER
(
ORDER BY
CASE
WHEN @OrderBy = 'Score' THEN Score
END DESC,
CASE
WHEN @OrderBy = 'Kills' THEN Kills
END DESC,
CASE
WHEN @OrderBy = 'Deaths' THEN Deaths
END DESC
) AS RowNum, *
FROM HeroTotals WHERE
(@Side is null or TeamId=@Side) and
(@Kit is null or Kit=@Kit)
)
AS RowConstrainedResult WHERE RowNum >= @Start AND RowNum < @Max

ORDER BY RowNum
Go to Top of Page

robvolk
Most Valuable Yak

15732 Posts

Posted - 2011-05-19 : 21:29:57
Table variables are fine most of the time, but 200,000 rows usually removes their advantages because they can't be indexed.

If you really only need to refresh once an hour, then it's no problem to stick with your current process. I figured you'd like to have the freshest data every time. If you can stand possible incorrect data, then NOLOCK will help prevent the SELECT from blocking the UPDATEs. Normally it should be avoided, but if you can't make it work otherwise it should be mostly safe to use.

The big reason I'm pushing to avoid 3 tables is that you're just storing 3 additional copies of the same data. This uses up RAM that could be used for the original table, and would cause more I/O if there's not enough RAM and it has to flush to disk.
Go to Top of Page
   

- Advertisement -