Monday, 16 June 2014

Optimising Index Optimisations on SQL Server


So you have created an maintenance plan for your databases which will rebuild indexes? Nice.
Is it taking a long time? Hmmm... Not so nice.

How can you improve this? Firstly, sometimes you can and should avoid messing with your indexes. Blindly rebuilding all indexes all of the time is a bit 'brute force' - you can however just get away with reorganising the index, and in other cases the index might not need any attention whatsoever. This is a more intelligent and optimised approach which could get you valuable time during a maintenance plan.
The sql code snippet below will 'intelligently' generate index optimising DDL. 

Remarks:
  1. Assumes that you know the difference between an index rebuild and an index reorganise.
  2. Avoid using DBCC DBREINDEX. It is obsolete and will be removed from a future version. (although still present in sql server 2014). Instead use ALTER INDEX.
  3. Using the Microsoft suggested fragmentation thresholds i.e. 
    1. <= 5% - dont do anything. The performance gains will be negligible compared relative to the cost of  rebuilding or reorganising.
    2. > 5% and <= 30% - perform a reorganise
    3. > 30% - perform a rebuild
  4. Assumes you are optimising a single database - i.e. the database where this sp will execute.
  5. Allows accuracy to be tweaked via @statsMode parameter.
  6. Assumes that you want to optimise all tables
  7. Assumes you have a sysadmin role on the server.
  8. Assumes you have dbo or db_ddladmin role on database.
  9. Assumes a constant fill factor for all tables e.g. 90%. Its a good starting point default but is it really the best setting for every index in your database? Probably not! Some tables will grow very fast and some wont grow at all. Meaning 90% will not be enough or be overkill respectively. Experiment by seeing how often you need to defrag each of your indexes. This suggests a further improvement for this sql - how to automatically decide good fill factors based on past fragmentation. This assumes you are regularly optimising indexes e.g. every day or every week and the database is growing at a fairly constant rate, then you could use the fragmentation percentage as a guide to 'predicting' what the fill factor should be during the next index rebuild. (you cant set fill factor during a reorganisation)
  10. In real life, I run the generated DDL statements script during my overnight maintenance plan.

IF  EXISTS (SELECT * FROM sys.objects WHERE OBJECT_ID = OBJECT_ID(N'[db_ddladmin].[spGenerateIndexOptimisationDDL]') AND TYPE IN (N'P', N'PC'))
DROP PROCEDURE [db_ddladmin].[spGenerateIndexOptimisationDDL]
GO

SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

-- =============================================
-- Author: Joe
-- Create date: 16/06/2014
-- Description: Build appropriate DDL to optimise indexes
-- =============================================
CREATE PROCEDURE [db_ddladmin].[spGenerateIndexOptimisationDDL]
@forceRebuild BIT = 0,
@forceReorganise BIT = 0,
@defaultFillFactor TINYINT = 90,
@statsMode nvarchar(20) = 'LIMITED' --(least accuracy, least time) LIMITED | SAMPLED | DETAILED (most accuray, most time)
AS
BEGIN

SET NOCOUNT ON;

if(@forceRebuild = 1)
begin
SELECT 
'ALTER INDEX [' + i.name + '] ON [' + SCHEMA_NAME(t.schema_id) + '].[' + t.name + '] REBUILD WITH (FILLFACTOR = ' + CAST(@defaultFillFactor AS VARCHAR(3)) + ', SORT_IN_TEMPDB = ON, STATISTICS_NORECOMPUTE = ON);'
FROM sys.indexes AS i
INNER JOIN sys.tables AS t ON t.object_id = i.object_id
WHERE i.name IS NOT NULL;
end
else if(@forceReorganise = 1)
begin
SELECT 
'ALTER INDEX [' + i.name + '] ON [' + SCHEMA_NAME(t.schema_id) + '].[' + t.name + '] REORGANIZE;'
FROM sys.indexes AS i
INNER JOIN sys.tables AS t ON t.object_id = i.object_id
WHERE i.name IS NOT NULL;
end
else
begin 
SELECT 
CASE 
WHEN s.avg_fragmentation_in_percent > 5.0 AND s.avg_fragmentation_in_percent <= 30.0 THEN 'ALTER INDEX [' + i.name + '] ON [' + SCHEMA_NAME(t.schema_id) + '].[' + t.name + '] REORGANIZE;'
WHEN s.avg_fragmentation_in_percent > 30.0 THEN 'ALTER INDEX [' + i.name + '] ON [' + SCHEMA_NAME(t.schema_id) + '].[' + t.name + '] REBUILD WITH (FILLFACTOR = ' + CAST(@defaultFillFactor AS VARCHAR(3)) + ', SORT_IN_TEMPDB = ON, STATISTICS_NORECOMPUTE = ON);'
ELSE NULL
END
FROM sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL, NULL, @statsMode) AS s
INNER JOIN sys.indexes AS i ON s.object_id = i.object_id AND s.index_id = i.index_id
INNER JOIN sys.tables AS t ON t.object_id = i.object_id
WHERE s.avg_fragmentation_in_percent > 5
AND i.name IS NOT NULL;
end
END;
GO

/* 
-- TESTING

EXEC [db_ddladmin].[spGenerateIndexOptimisationDDL] 
@forceRebuild = 0,
@forceReorganise = 0,
@defaultFillFactor = 90,
@statsMode = 'SAMPLED' --(least accuracy, least time) LIMITED | SAMPLED | DETAILED (most accuray, most time)
*/



References:
http://msdn.microsoft.com/en-gb/library/ms189858.aspx

Thursday, 15 May 2014

SQL Server Intellisense Not Working?

Im using SQL Server 2008 - or to be exact (as sometimes these things make a difference...:)

Microsoft SQL Server 2008 (SP3) - 10.0.5512.0 (X64)   Aug 22 2012 19:25:47   Copyright (c) 1988-2008 Microsoft Corporation  Developer Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1.

With that said, you should check the following:

  1. With an open query in SSMS > Edit > Intellisense > Refresh Local Cache (Ctrl + Shift + R)
  2. SSMS > Tools > Options > Text Editor > Transact-SQL > General : > Statement completion:
    1. auto list members = checked
    2. parameter information = checked
  3. SSMS > Tools > Options > Text Editor > Transact-SQL > IntelliSense > Enable IntelliSense
  4. With an open query in SSMS > Query > Intellisense Enabled (should already be enabled because of point 3 but just in case...)
  5. With an open query in SSMS > Query > SQLCMD Mode should be disabled (if you want intellisense to work!) This is what caught me out recently and prompted this post. I had set my queries set to open in SQLCMD mode by default i.e. SMS > Tools > Options > Query Execution > SQL Server > General > 'By default, open new queries in SQLCMD mode' = checked.
If its still not working then try the latest services packs, cumulative updates, installation repair or just turn it off and on again. Good luck. Happy sensing!

Wednesday, 12 October 2011

Visual Studio Not Showing GAC Assemblies

While using Visual Studio 2010 i recently noticed that i couldnt see references to the Microsoft.SQLServer set of dlls.
I could see them if I went to C:\Windows\assembly.
They were definitely installed - and i could manually reference them by browsing to:
C:\Program Files\Microsoft SQL Server\100\SDK\Assemblies\

And in fact my project was using them...
Other websites suggested that they should be listed under the .Net tab when adding references.
I checked the registry:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework
and under AssemblyFolders i had the key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\AssemblyFolders\SQL Server Assemblies - with the value: C:\Program Files\Microsoft SQL Server\100\SDK\Assemblies\

I then found this article:
See section "To display an assembly in the Add Reference dialog box"

So i tried adding this key to the registry:


HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319\AssemblyFoldersEx\SQL Server Assemblies

Restarted visual studio, and now my .Net assemblies are back under the .Net tab...

UPDATE:

I have just installed the DirectX 9 with managed code support SDK...
the same problem. I was able to find this:
http://connect.microsoft.com/VisualStudio/feedback/details/464577/

"The underlying issue is that AssemblyFolders is not multi-targeting aware and so does not work well with our new multi-targeting features in VS 2010. AssemblyFoldersEx is version specific and so works with multi-targeting.
If you have a control that works with multiple versions of the .Net Framework you register it with the lowest version of the framework that it works with. So for example if you have a control that works with .Net 3.0 and up you would register it under the .Net 3.0 entry of AssemblyFoldersEx. It would then show up in the Choose Items and Add References dialog for 3.projects targeting 3.0, 3.5 and 4.0 but not projects targeting 2.0. "

The libraries that i have been trying to install are not aware of this and so just use the old registry location

Wednesday, 3 November 2010

Debugging code started from the Global.asax Application_Start event when using IIS 7 integrated pipeline

The new integrated pipeline with IIS7 is good... I think!? what I mean is that I know its good under the bonnet but it really doesn't affect me much once my website is running. So lets assume that we have converted our web.config to use integrated pipeline and everything is working fine... (this alone can take a while, but that's another blog...)

Then one day you need to step into some code that occurs whenever the Application_Start event fires from Global.asax... You find that even though you have breakpoints in the code, they appear to be ignored and the application starts up. The trick to getting into this code is...

  1. Run the website from visual studio - allow it to start normally - your breakpoints will be ignored as usual.
  2. Edit the web.config file - just adding a space character and then re-saving will do - you don't actually have to change any settings. 
  3. Editing the web.config in any way will cause the application to stop.
  4. Make a request to the website e.g. request a page, web service etc. This will cause the application to start.
  5. Because we are ALREADY attached to the process with the visual studio debugger, we will now get to step into the application_start event. 
  6. debugging joy!
I believe the reason we cant step into the application_start on normal visual studio debug startup is because by the time we have attached to the website process, the application_start event has already occurred.

This solution may not work if you are unable to make the website run in the first place, i.e. if the code in application_start doesn't allow the website to run because it is faulty/buggy etc.

Solutions that didnt work but looked good at the time...

  1. Visual Studio > Web site project property pages > Start Options > Start Action > Select "Dont open a page. Wait for a request from an external application" - I thought this would perhaps delay the first request to IIS and therefore delay application_start - meaning that when we do request a page, our debugger would be attached. This doesnt work though as the application will start regardless of this option. There must be another request sent from visual studio during the start up of the debugger. Todo: we should really investigate this by analysing the requests made to IIS when we start the debugger. There may be another setting?
  2. Recycling the application pool associated with the website you are debugging - unfortunately although this does cause application_event to fire, it also causes visual studio to detach from the process! No stepping in goodness to be had here :(




Wednesday, 27 October 2010

Visual Studio auto formatting of XML attributes

Visual Studio 2010 > Tools > Options > Text Editor > XML > Formatting > Attributes > Select 'Align attributes each on a separate line'.

After this is selected it will be possible to use the visual studio chord sequence: 'ctrl k' + 'ctrl d'.

This will auto format the attributes in an XML file so that each attribute has a new line. I find XML files easier to read when formatted like this. It is also easier to visually compare 2 XML files that have this formatting. (e.g. using ExamDiff)