Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
PedroSQL
Starting Member
2 Posts |
Posted - 2011-05-19 : 17:10:16
|
Dear sql gurus,I have a system that collects game scores from each round & totals them in a PlayerTotals table.The PlayerTotals table is an average 200,000 rows. It is constantly being updated with new scores and players.I need to query the table, for a scoreboard, which can be sorted by score, kills & deaths.Querying the live table whilst it is also being updated is presumably bad. I decided to create duplicate tables that have already been sorted (PlayerTotals_Score, PlayerTotals_Kills, PlayerTotals_Deaths). I plan to run the updates every hour, then the scoreboard queries THESE tables instead of the 'live' table (paging, find position of specific player, etc)Below is my initial attempt. Ultimately I'm trying to produce fast sorted/paged/filtered scores, and not affect update speed of the original source.1) Am I correct that I should do this away from the 'high volume update' table PlayerTotals?2) Should I be puling this to another database (on same server) to process?3) Is there just a much better way of doing all this?All advice or just random suggestions welcomed!PS. I presume I should be locking these tables from the website whilst deleting and refilling?[edit] Code below took 3 minutes to run the first time, and just hung the server the second time :(
DECLARE @Temp TABLE ( [UserId] [bigint] NOT NULL, [CharacterName] [varchar](30) NOT NULL, [Kills] [int] NOT NULL, [Deaths] [int] NOT NULL, [Score] [int] NOT NULL, [LastSeen] [datetime] NOT NULL, [TeamId] [int] NOT NULL, [GamesPlayed] [int] NULL, [Kit] [nchar](100) NULL, [DayStartPosition] [int] NULL, [KDR] [real] NULL)insert into @Temp (UserId,CharacterName,Kills,Deaths,Score,LastSeen,TeamId,GamesPlayed,Kit,DayStartPosition,KDR) SELECT * FROM PlayerTotalsdelete from PlayerTotals_Scoreinsert into PlayerTotals_Score (RowId,UserId,CharacterName,Kills,Deaths,Score,LastSeen,TeamId,GamesPlayed,Kit,DayStartPosition,KDR)SELECT * FROM ( SELECT ROW_NUMBER() OVER (ORDER BY Score DESC) AS RowNum, * FROM @Temp) AS RowConstrainedResultORDER BY RowNum delete from PlayerTotals_Killsinsert into PlayerTotals_Kills (RowId,UserId,CharacterName,Kills,Deaths,Score,LastSeen,TeamId,GamesPlayed,Kit,DayStartPosition,KDR)SELECT * FROM ( SELECT ROW_NUMBER() OVER (ORDER BY Kills DESC) AS RowNum, * FROM @Temp) AS RowConstrainedResultORDER BY RowNum delete from PlayerTotals_Deathsinsert into PlayerTotals_Deaths (RowId,UserId,CharacterName,Kills,Deaths,Score,LastSeen,TeamId,GamesPlayed,Kit,DayStartPosition,KDR)SELECT * FROM ( SELECT ROW_NUMBER() OVER (ORDER BY Deaths DESC) AS RowNum, * FROM @Temp) AS RowConstrainedResultORDER BY RowNum |
|
|
robvolk
Most Valuable Yak
15732 Posts |
Posted - 2011-05-19 : 18:16:41
|
quote: Querying the live table whilst it is also being updated is presumably bad.
Why presume that? You're querying it to populate the table variable.I'd suggest a few things (assuming you haven't tried them yet):1. Create indexes on the Score, Kills, and Deaths columns on the PlayerTotals table and query it directly with the correct ORDER BY clause.2. If #1 not possible, don't use a table variable. Insert the data into a temp table and create the indexes as described.3. Use a single temp table instead of 3.4. Limit the number of rows returned in the query, like TOP 100 or TOP 1000 at most. Use ROW_NUMBER() to navigate/return pages of data only if requested (i.e. not Page 1).5. Consider NOLOCK and FAST query hints if the UPDATE query is being impacted. |
 |
|
|
PedroSQL
Starting Member
2 Posts |
Posted - 2011-05-19 : 19:14:24
|
Thank you for your suggestions robvolk.I thought I should move to a table variable to prevent the rest of the queries from holding up the table longer than needed. Is that a false assumption? I've had a couple of painful server freezes whilst testing this stuff.1) I will read up more on indexes.2) I read table variables are quicker than temp tables?3) These are not temp tables. They are 'cache' tables that the website uses, so the grunt work of ordering and adding rownum is done just once an hour. I figured running the type of query below would be even faster without the ordering. Most scoreboards I see online are updated periodically, presumably for these kind of query speed/lock-up reasons?4) You're absolutely right about limiting to 1,000 rows, silly me. It now takes just 13 seconds to run.5) I will read up more on NOLOCK and FASTI just assumed there was a better way to do things rather than have a frequently visited website querying the same table that hundreds of servers are updating.The main thing I want to avoid is any lock-ups, as I can only check in on things each evening after work, and don't want to find it's been hung all day.My original 'direct' multi parameter, paged, filtered query is below, in case it helps anyone else. It actually only takes 1 second to run on a 200,000 record table.CREATE PROCEDURE [dbo].[spGetFilteredScoreboard]@Start int = null, @Side int = null,@Kit varchar(30) = null, @Name varchar(30) = null ,@OrderBy varchar(30)AS declare @Max Integerdeclare @Position Integerif (@Name is not null)begin set @Position = (SELECT RowNum FROM ( SELECT ROW_NUMBER() OVER ( ORDER BY CASE WHEN @OrderBy = 'Score' THEN Score END DESC, CASE WHEN @OrderBy = 'Kills' THEN Kills END DESC, CASE WHEN @OrderBy = 'Deaths' THEN Deaths END DESC ) AS RowNum, * FROM HeroTotals WHERE (@Side is null or TeamId=@Side) and (@Kit is null or Kit=@Kit)) AS RowConstrainedResult WHERE CharacterName = @Name) set @Start = @Position - 50endif (@Start < 1) set @Start = 1set @Max = @Start + 100;SELECT * FROM ( SELECT ROW_NUMBER() OVER ( ORDER BY CASE WHEN @OrderBy = 'Score' THEN Score END DESC, CASE WHEN @OrderBy = 'Kills' THEN Kills END DESC, CASE WHEN @OrderBy = 'Deaths' THEN Deaths END DESC) AS RowNum, * FROM HeroTotals WHERE (@Side is null or TeamId=@Side) and (@Kit is null or Kit=@Kit) ) AS RowConstrainedResult WHERE RowNum >= @Start AND RowNum < @Max ORDER BY RowNum |
 |
|
|
robvolk
Most Valuable Yak
15732 Posts |
Posted - 2011-05-19 : 21:29:57
|
| Table variables are fine most of the time, but 200,000 rows usually removes their advantages because they can't be indexed.If you really only need to refresh once an hour, then it's no problem to stick with your current process. I figured you'd like to have the freshest data every time. If you can stand possible incorrect data, then NOLOCK will help prevent the SELECT from blocking the UPDATEs. Normally it should be avoided, but if you can't make it work otherwise it should be mostly safe to use.The big reason I'm pushing to avoid 3 tables is that you're just storing 3 additional copies of the same data. This uses up RAM that could be used for the original table, and would cause more I/O if there's not enough RAM and it has to flush to disk. |
 |
|
|
|
|
|
|
|