Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
derach2000
Starting Member
37 Posts |
 Posted - 2009-12-07 : 06:01:33
|
| Hello,I'm new here so I apologize if this is not the best place to ask this question.I'm still using mssql2000 but an upgrade is due any time now.I have a problem with periodical data update. I have one table that gathers all lot of data from various sources, from which I need to create and update a statistical table.Is there an easy way to implemet this or do have to write a massive stored procedure using cursors.Kind regards |
|
|
RyanRandall
Master Smack Fu Yak Hacker
1074 Posts |
Posted - 2009-12-08 : 07:11:37
|
| It depends on your data and on the statistics you want from it.If you give us the structure of your table, some sample data, and the output you'd like based on that sample data, we'll be able to advise you better.Ryan Randall - Yak of all tradesSolutions are easy. Understanding the problem, now, that's the hard part. |
 |
|
|
derach2000
Starting Member
37 Posts |
Posted - 2009-12-08 : 08:08:32
|
| Hi,Sorry for not giving any details.The first table is quite big, it has over 70 columns. Now, not every of them are used. Most of those being used are decimals (9,2).There will be bulk inserts to that table,possibly every half hour, not real time.There should be at least 35000 records daily, and that is only one source. Number of sources should go up to 100 or even more, to 1000 in some rare cases.So we are talking abount millions or records on a larger time scale, say years and such.There will come a time when it it is going to be necessary to archive and truncate this table, when - don't know.The statitical table will include summarizing most of the columns grouped by days and source.We are inclined to perform thees statistical calculations once a day.Curently we use our application to go threw the table.We think about using transaction during every row update becouse we mark every row that was used for calculations, update results in stat tables, ending the transaction.So, there are going to be a lot of rows. What would be the best practice concerning bulk inserts and calculation speed, or what should be kept in mind when developing and testing such an eviroment.I hope you can gain some insigth concernig my problem.Kind regards |
|
|
|
|
|
|
|
|
Periodical data update