After the tip https://www.mssqltips.com/sqlservertip/5526/how-to-setup-and-use-a-sql-server-stretch-database/ I made a video to show step by step how to enable stretch database.
All system databases, except the resource database, can be moved to new locations to help balance I/O load.
To move the msdb, model, and tempdb databases, perform the following steps:
- For each file to be moved, execute the ALTER DATABASE … MODIFY FILE statement.
- Stop the instance of SQL Server.
- Move the files to the new location (this step is not necessary for tempdb, as its files are recreated automatically on startup).
- Restart the instance of SQL Server.
The process for moving the master database is different from the process for other databases. To move the master database, perform the following steps:
- Open SQL Server Configuration Manager.
- In the SQL Server Services node, right-click the instance of SQL Server, click Properties, and then click the Startup Parameters tab.
- Edit the Startup Parameters values to point to the planned location for the master database data (-d parameter) and log (-l parameter) files.
- Stop the instance of SQL Server.
- Move the master.mdf and mastlog.ldf files to the new location.
- Restart the instance of SQL Server
Parallelism refers to multiple processors cooperating to execute a single query at the same time.
Parallel execution involves the overhead of synchronizing and monitoring the tasks. So, that’s why parallel plans are considered expensive operations.
The query optimizer determines whether a parallel plan should be used based on the configuration and the query cost.
- A configuration value, max degree of parallelism determines how many CPUs can be used to execute a query. Also, using query hint can set how many CPUs can be used for that specific query.
- Cost threshold for parallelism determines the cost that a query must meet before a parallel query plan will even be considered. Query cost is determined based on the amount of data the query optimizer estimates to be read to complete the operation.
In earlier versions of SQL Server, it was common to disable parallel queries on systems that were primarily used for transaction processing. It’s common to see MAXDOP 1 for OLTP environments. I prefer to raise the cost threshold for parallelism so a parallel plan is only considered for higher cost queries.
Let’s see an example.
I executed the query below and I got parallelism for that. My query cost was 374.148, remember this cost is always estimated.
SELECT * FROM dbo.FactInternetSales AS FIS LEFT JOIN dbo.DimCustomer AS DC ON DC.CustomerKey = FIS.CustomerKey LEFT JOIN dbo.DimProduct AS DP ON DP.ProductKey = FIS.ProductKey LEFT JOIN dbo.DimProductSubcategory AS DPS ON DPS.ProductSubcategoryKey = DP.ProductSubcategoryKey ORDER BY DP.StandardCost OPTION(RECOMPILE)
Increasing the cost threshold for parallelism value to 400 you are not going to see any change in the query, because the real cost isn’t 374, that cost is for only 1 thread.
In my case the max degree of parallelism value is 2, so multiplying 374 x 2 = 748
Setting the value to 800 and run the same query again you can see no parallelism and the cost is 691.208, so it fits in the cost value.
If your CXPACKET wait counter is high, try to increase the cost threshold for parallelism and not set MAXDOP to 1 because some queries even in a transaction database will have the benefit to run in parallel.
Doing crash and recovery tests on my local machine I got the SQL Server instance not going online. After trying the third time to bring my instance online thinking was something else problem I saw the SQL Server errolog file and I could see the problem.
Not enough memory, but wasn’t on my machine, was in the resource pool. So, what is a resource pool?
A resource pool represents a subset of the physical resources of an instance of the Database Engine and in my case was insufficient memory. Let’s see the errorlog file:
2018-03-14 16:19:58.09 spid56s [ERROR] Recovery failed with error 0x83000000 on database 18. This error will be mapped to 'HK_E_RESTORE_INSUFFICIENT_MEMORY' (0x8200002e). (sql\ntdbms\hekaton\runtime\src\hkruntime.cpp : 4805 - 'HkRtRestoreDatabase') 2018-03-14 16:19:58.09 spid34s [INFO] HkCkptCtrlUninitialize(): Database ID: . Cleaning up StorageArray. LastClosedCheckpointEndTs: '158' 2018-03-14 16:19:58.09 Server Error: 17300, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped. 2018-03-14 16:19:58.09 Server Error: 17312, Severity: 16, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped. 2018-03-14 16:19:58.09 Server Error: 28709, Severity: 16, State: 19. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped. 2018-03-14 16:19:58.09 spid37s Error: 701, Severity: 17, State: 137. 2018-03-14 16:19:58.09 spid37s There is insufficient system memory in resource pool 'default' to run this query. 2018-03-14 16:19:58.09 spid39s Error: 701, Severity: 17, State: 137. 2018-03-14 16:19:58.09 spid39s There is insufficient system memory in resource pool 'default' to run this query. 2018-03-14 16:19:58.11 spid55s [ERROR] Recovery failed with error 0x83000000 on database 15. This error will be mapped to 'HK_E_RESTORE_INSUFFICIENT_MEMORY' (0x8200002e). (sql\ntdbms\hekaton\runtime\src\hkruntime.cpp : 4805 - 'HkRtRestoreDatabase') 2018-03-14 16:19:58.11 spid31s [INFO] HkCkptCtrlUninitialize(): Database ID: . Cleaning up StorageArray. LastClosedCheckpointEndTs: '155' 2018-03-14 16:19:58.20 spid31s SQL Server shutdown has been initiated 2018-03-14 16:19:58.21 spid31s Error: 19032, Severity: 10, State: 1. (Params:). The error is printed in terse mode because there was error during formatting. Tracing, ETW, notifications etc are skipped. 2018-03-14 16:19:58.28 spid34s SQL Server shutdown has been initiated
After starting the service SQL Server was doing the redo and undo process, this means it was reading the log files, create the compensate log records if was found any uncommitted transaction.
SQL Server will need memory in buffer pool to complete the redo and undo process and I didn’t remember I changed any SQL Server memory configuration.
So, my approach was to connect SQL Server via command line while the instance was still up and run sp_configure to see how much memory was configured. I got only 512mb set for Max Server Memory and that was the problem. (Max server memory controls the SQL Server memory allocation, compile memory, all caches (including the buffer pool), query execution memory grants, lock manager memory, and CLR memory).
In my environment with 26 databases and my crash recovery tests, 512mb for my pool memory wasn’t enough and when I changed the configuration to 4096mb I could bring the instance online again.
First, read the errorlog file to have more information what SQL Server is doing and also know transaction log operations, log records, checkpoints and how crash recovery works is fundamental.
SQL Server as a process acquires more memory than specified by max server memory option. Both internal and external components can allocate memory outside of the buffer pool, which consumes additional memory, but the memory allocated to the buffer pool usually still represents the largest portion of memory consumed by SQL Server.
I like tools and I’ve used a couple of them. I had the opportunity to try SQL Examiner to compare database structures and even though compare data. The tool is quite simple to configure the options.
Talking about comparison, well we compare one database with another. Imagine a situation, there are two environments (development and test) and we must deploy the changes from one to another.
Let’s see step by step how it works
The steps are quite simple, need to select source and target type (Microsoft SQL Server, Oracle, MySQL, PostgreSQL, SQL Azure). After that, define the servers name, the authentication mode and the database. Below I’m showing how to connect in 2 local instances I have on my local computer.
After that, the SQL Examiner will compare what differences exist between databases.Now, I’d say I was expecting my objects differences and I got a great feature. In the image below the SQL Examiner is comparing the database options as well.
This is fantastic, the tool is showing different compatibility levels and it can be very helpful if you are doing performance tests, for example, the behavior is very different in this case. Also, I can compare my production environment and keep the database configurations equal in all environments.
Great tool, simple and efficient.
The first checklist before start using any application using SQL Server should be the default configurations.
What configurations I used to change?
- Max Server Memory: SQL Server will grab all memory available and that will cause concurrency with the OS
- MAXDOP: for OLTP also yes, the good thing is to use it
- Cost Threshold for Parallelism: the default value is too low, and the number will depend on the environment, I used to monitor and change as needed. I use to start with 40
- Model File Sizes: the model database will provide information when need to create more databases. (initial size and autogrowth are way too small and need to avoid file system fragmentation)
- Tempdb Files: look how many CPUs in the server and if the number is less than 8, so would be the number of CPUs, if it is more, so, start with 8 and the files should be the same size and the same autogrowth.
- Database Mail: used to send alerts, very important to get notified when something bad happens.
- Backup Compression: the benefit will be much more than the CPU cost.
- Remote Dedicated Administrator Connection: provides dedicated CPU, memory, and scheduler and by default only works via RPD or physically on server
- Maintenance: start as soon as possible with checkdb, index maintenance, statistics, backups. In this case, I like Ola Hallengren SQL Server solution.
- The last but not least, Security: this is the most important in the list, user accounts or even DBAs as sysadmin, look my tip in MSSQLTips. In addition, disable sa, disable unnecessary logins.