Install dbatools module without admin rights

If you have been face a issue that you don’t have rights to install any powershell module like showing the image below for dbatools, one workaround is to install it only for your user.

In this case you can use the scope parameter to install it in your user only as you can see in the message.

Install-Module dbatools -Scope currentuser



How to add Active Directory powershell module

I’m going to show the steps how to add the AD module in your machine and in the server if you need to get information like run Get-ADPrincipalGroupMembership to know what groups a specific user belongs.

To add this module in your local machine is quite easy. Install the Remote Server Administration Tools for Windows 10 package

But, to install it in a Windows Server for example you need to add this feature following the steps .

  1. Open Server Manager Dashboard
  2. Click Manage -> Add Roles and Features Wizard
  3. Click next until to show the picture below in the item features

After install it (no restart needed) you can run import-module ActiveDirectory and run something like this:

Get-ADPrincipalGroupMembership -Identity 'user_name' | Select-Object name

$groups = Get-ADGroup -Filter {name -like 'user_group'} -Properties * | Select -property name

foreach($group in $groups) {
    if($group.name -ceq 'Group trying to find') {
        $group.name
        Get-ADGroupMember -Identity $group.name | Where-Object objectClass -Like 'group' | Select-Object name
    }
}

T-SQL script to get detailed login permissions

This script is very helpful when you need to know all the permissions a user have in detail.

 

SELECT distinct 
@@SERVERNAME as ServerName
, UserName as LoginName
, UserType as LoginType
, DatabaseUserName
, Role
, PermissionType
, PermissionState
, DatabaseName = db_name()
, ObjectName  -- can be an object or the entire database
, ObjectType
, ColumnName
--	, item -- used to check consistencies in each subquery
FROM (
SELECT  
    UserName = princ.name,
    UserType = CASE princ.type
                    WHEN 'S' THEN 'SQL'
                    WHEN 'U' THEN 'Windows'
					WHEN 'G' THEN 'Windows Group'
					WHEN 'R' THEN 'Database Role'
					ELSE princ.type 
                END,  
    DatabaseUserName = princ.name,       
    Role = princ.name,      
    PermissionType = perm.permission_name,       
    PermissionState = perm.state_desc,       
    ObjectType = obj.type_desc,
    ObjectName = coalesce(OBJECT_NAME(perm.major_id), db_name()),
    ColumnName = col.name,
	1 as item
FROM    
    sys.database_principals princ  
LEFT JOIN sys.login_token ulogin on princ.sid = ulogin.sid
LEFT JOIN sys.database_permissions perm ON perm.grantee_principal_id = princ.principal_id
LEFT JOIN sys.columns col ON col.object_id = perm.major_id AND col.column_id = perm.minor_id
LEFT JOIN sys.objects obj ON perm.major_id = obj.object_id
WHERE perm.permission_name <> 'CONNECT' 
UNION
SELECT  
    UserName =  memberprinc.name ,
    UserType = CASE memberprinc.type
                    WHEN 'S' THEN 'SQL'
                    WHEN 'U' THEN 'Windows'
					WHEN 'G' THEN 'Windows Group'
					WHEN 'R' THEN 'Database Role'
					else memberprinc.type 
                END, 
    DatabaseUserName = memberprinc.name,   
    Role = roleprinc.name,      
    PermissionType = perm.permission_name,       
    PermissionState = perm.state_desc,       
    ObjectType = obj.type_desc,
    ObjectName = db_name(),
    ColumnName = col.name,
	2 as item
FROM    
    sys.database_role_members members
INNER JOIN sys.database_principals roleprinc ON roleprinc.principal_id = members.role_principal_id
INNER JOIN sys.database_principals memberprinc ON memberprinc.principal_id = members.member_principal_id
LEFT JOIN sys.login_token ulogin on memberprinc.sid = ulogin.sid
LEFT JOIN sys.database_permissions perm ON perm.grantee_principal_id = roleprinc.principal_id
LEFT JOIN sys.columns col on col.object_id = perm.major_id AND col.column_id = perm.minor_id
LEFT JOIN sys.objects obj ON perm.major_id = obj.object_id
UNION
SELECT  
    UserName = roleprinc.name,
    UserType = 
	CASE roleprinc.type
                    WHEN 'S' THEN 'SQL'
                    WHEN 'U' THEN 'Windows'
					WHEN 'G' THEN 'Windows Group'
					WHEN 'R' THEN 'Database Role'
					ELSE roleprinc.type 
                END,  
    DatabaseUserName = roleprinc.name,
    Role = roleprinc.name,      
    PermissionType = perm.permission_name,       
    PermissionState = perm.state_desc,       
    ObjectType = obj.type_desc,
    ObjectName = coalesce(OBJECT_NAME(perm.major_id), db_name()),
    ColumnName = col.name,
	3 as item
FROM    
    sys.database_principals roleprinc 
LEFT JOIN sys.database_permissions perm ON perm.grantee_principal_id = roleprinc.principal_id
LEFT JOIN sys.columns col on col.object_id = perm.major_id AND col.column_id = perm.minor_id                   
LEFT JOIN sys.objects obj ON obj.object_id = perm.major_id
UNION
SELECT 
    UserName = princ.name collate Latin1_General_CI_AS,
    UserType = CASE princ.type
                    WHEN 'S' THEN 'SQL'
                    WHEN 'U' THEN 'Windows'
					WHEN 'G' THEN 'Windows Group'
					WHEN 'R' THEN 'Database Role'
					ELSE princ.type 
                END ,  
    DatabaseUserName = princ.name collate Latin1_General_CI_AS,
    Role =  CASE 
                WHEN logins.sysadmin = 1 THEN 'sysadmin'
                WHEN logins.securityadmin = 1 THEN 'securityadmin'
                WHEN logins.serveradmin = 1 THEN 'serveradmin'
                WHEN logins.setupadmin = 1 THEN 'setupadmin'
                WHEN logins.processadmin = 1 THEN 'processadmin'
                WHEN logins.diskadmin = 1 THEN 'diskadmin'
                WHEN logins.dbcreator = 1 THEN 'dbcreator'
                WHEN logins.bulkadmin = 1 THEN 'bulkadmin'
                ELSE 'Public' 
            END,
    PermissionType	= perm.permission_name,
    PermissionState = 'GRANT',
    ObjectType		= NULL,
    ObjectName		= princ.default_database_name,
    ColumnName		= NULL,
	4 as item
FROM sys.server_principals princ 
INNER JOIN sys.syslogins logins ON princ.sid = logins.sid 
LEFT JOIN sys.database_permissions perm ON perm.grantee_principal_id = princ.principal_id
WHERE princ.type  <> 'R' AND princ.name NOT LIKE '##%'
) P  
where (Role <> 'Public' or ObjectName = db_name())
ORDER BY
P.DatabaseUserName,
P.ObjectName,
P.ColumnName,
P.PermissionType,
P.PermissionState,
P.ObjectType

Using wait stats on SQL Server

I wrote two posts about wait stats:

  1.  What wait means
  2. About SQL Server saving wait stats on DMVs

Now, going deeper on wait stats and see “why SQL Server is running slow?”. To answer that question I like to start with the DMV sys.dm_os_wait_stats, because this DMV provides a running total of all waits encontered by executing threads in SQL Server instance.

SQL Server categorizes waits across several different type and some of these types only indicate quit period on the instance where threads stay in waiting.

The script below shows the top wait types that have accumulated since SQL Server started or was cleared.

WITH Waits AS (
SELECT  wait_type
    , CAST(wait_time_ms / 1000. AS DECIMAL(12, 2)) AS [wait_time_s]
    , CAST(100. * wait_time_ms / SUM(wait_time_ms) OVER () AS DECIMAL(12, 2)) AS [pct]
    , ROW_NUMBER() OVER (ORDER BY wait_time_ms DESC) AS rn 
FROM sys.dm_os_wait_stats WITH (NOLOCK) 
WHERE  
     wait_type NOT IN (N'CLR_SEMAPHORE', N'LAZYWRITER_SLEEP', N'RESOURCE_QUEUE', N'SLEEP_TASK', N'SLEEP_SYSTEMTASK', N'SQLTRACE_BUFFER_FLUSH', N'WAITFOR', N'LOGMGR_QUEUE', N'CHECKPOINT_QUEUE', N'REQUEST_FOR_DEADLOCK_SEARCH', N'XE_TIMER_EVENT', N'BROKER_TO_FLUSH', N'BROKER_TASK_STOP', N'CLR_MANUAL_EVENT', N'CLR_AUTO_EVENT', N'DISPATCHER_QUEUE_SEMAPHORE', N'FT_IFTS_SCHEDULER_IDLE_WAIT', N'XE_DISPATCHER_WAIT', N'XE_DISPATCHER_JOIN', N'SQLTRACE_INCREMENTAL_FLUSH_SLEEP', N'ONDEMAND_TASK_QUEUE', N'BROKER_EVENTHANDLER', N'SLEEP_BPOOL_FLUSH', N'SLEEP_DBSTARTUP', N'DIRTY_PAGE_POLL', N'HADR_FILESTREAM_IOMGR_IOCOMPLETION', N'SP_SERVER_DIAGNOSTICS_SLEEP', N'QDS_PERSIST_TASK_MAIN_LOOP_SLEEP', N'QDS_CLEANUP_STALE_QUERIES_TASK_MAIN_LOOP_SLEEP', N'WAIT_XTP_HOST_WAIT', N'WAIT_XTP_OFFLINE_CKPT_NEW_LOG', N'WAIT_XTP_CKPT_CLOSE', N'PWAIT_ALL_COMPONENTS_INITIALIZED')), Running_Waits AS (
SELECT  W1.wait_type
    , wait_time_s
    , pct
    , SUM(pct) OVER (ORDER BY pct DESC ROWS UNBOUNDED PRECEDING) AS [running_pct] 
FROM   Waits AS W1)
SELECT     
    wait_type
    , wait_time_s
    , pct
    , running_pct
FROM      Running_Waits
WHERE     running_pct - pct = 99
ORDER BY  running_pct
OPTION (RECOMPILE);

This query I got from Paul Randal blog, his blog has a lot of information about waits. Since not all wait types are indicators of real issue, the where clause is removing unnecessary type.

The common wait types for me are in the table below, there are much more, but you can start with that list.

Wait_TypeAreaDescriptionAction
ASYNC_IO_COMPLETIONI/OUsed to indicate a worker is waiting on a asynchronous I/O operation to complete not associated with database pagesSince this is used for various reason you need to find out what query or task is associated with the wait. Two examples of where this wait type is used is to create files associated with a CREATE DATABASE and for “zeroing” out a transaction log file during log creation or growth.
CHECKPOINT_QUEUEBufferUsed by background worker that waits on events on queue to process checkpoint requests. This is an “optional” wait type see Important Notes section in blogYou should be able to safely ignore this one as it is just indicates the checkpoint background worker is waiting for work to do. I suppose if you thought you had issues with checkpoints not working or log truncation you might see if this worker ever “wakes up”. Expect higher wait times as this will only wake up when work to do
CHKPTBufferUsed to coordinate the checkpoint background worker thread with recovery of master so checkpoint won’t start accepting queue requests until master onlineYou should be able to safely ignore. You should see 1 wait of this type for the server unless the checkpoint worker crashed and had to be restarted.. If though this is technically a “sync” type of event I left its usage as Background
CXPACKETQueryUsed to synchronize threads involved in a parallel query. This wait type only means a  parallel query is executing.You may not need to take any action. If you see high wait times then it means you have a long running parallel query. I would first identify the query and determine if you need to tune it. Note sys.dm_exec_requests only shows the wait type of the request even if multiple tasks have different wait types. When you see CXPACKET here look at all tasks associated with the request. Find the task that doesn’t have this wait_type and see its status. It may be waiting on something else slowing down the query. wait_resource also has interesting details about the tasks and its parallel query operator
IO_COMPLETIONI/OUsed to indicate a wait for I/O for operation (typically synchronous)  like sorts and various situations where the engine needs to do a synchronous I/OIf wait times are high then you have a disk I/O bottleneck. The problem will be determining what type of operation and where the bottleneck exists. For sorts, it is on the storage system associated with tempdb. Note that database page I/O does not use this wait type. Instead look at PAGEIOLATCH waits.
LAZYWRITER_SLEEPBufferUsed by the Lazywriter background worker to indicate it is sleeping waiting to wake up and check for work to doYou should be able to safely ignore this one. The wait times will appear to “cycle” as LazyWriter is designed to sleep and wake-up every 1 second. Appears as LZW_SLEEP in Xevent
LOGBUFFERTransaction LogUsed to indicate a worker thread is waiting for a log buffer to write log blocks for a transactionThis is typically a symptom of I/O bottlenecks because other workers waiting on WRITELOG will hold on to log blocks. Look for WRITERLOG waiters and if found the overall problem is I/O bottleneck on the storage system associated with the transaction log
RESOURCE_SEMAPHOREQueryUsed to indicate a worker is waiting to be allowed to perform an operation requiring “query memory” such as hashes and sortsHigh wait times indicate too many queries are running concurrently that require query memory. Operations requiring query memory are hashes and sorts. Use DMVs such as dm_exec_query_resource_semaphores and dm_exec_query_memory_grants
SOS_SCHEDULER_YIELDSQLOSUsed to indicate a worker has yielded to let other workers run on a schedulerThis wait is simply an indication that a worker yielded for someone else to run. High wait counts with low wait times usually mean CPU bound queries. High wait times here could be non-yielding problems
THREADPOOLSQLOSIndicates a wait for a  task to be assigned to a worker threadLook for symptoms of high blocking or contention problems with many of the workers especially if the wait count and times are high. Don’t jump to increase max worker threads especially if you use default setting of 0. This wait type will not show up in sys.dm_exec_requests because it only occurs when the task is waiting on a worker thread. You must have a worker to become a request. Furthermore, you may not see this “live” since there may be no workers to process tasks for logins or for queries to look at DMVs.
WRITELOGI/OIndicates a worker thread is waiting for LogWriter to flush log blocks.High waits and wait times indicate an I/O bottleneck on the storage system associated with the transaction log

The next posts I will show an example for each wait in the list.

Prost!

Using SQL Operations Studio Agent

SQL Operations Studio Agent is visual way to see, edit and create SQL Server jobs and everybody was asking this feature when SQL Operations Studio appears.

To install this the agent follow the link https://docs.microsoft.com/en-us/sql/sql-operations-studio/sql-server-agent-extension?view=sql-server-2017.

sqlops01

Installing the agent 

sqlops02

With the extension installed, right-click on the instance -> Manage to see the server jobs 

sqlops03

But, not everything is working. I tried to edit a job step and didn’t work. For example, the Ola hallengren maintenace scripts after created wasn’t possible to change using the GUI differently than using SSMS.

Even trying to using the GUI to create a new step didn’t work and I changed using TSQL.

sqlops05
sqlops04

The script is available in my Git here to download. After changed the script I succeeded ran it.

With extensions we have more features available to work with SQL Operations Studio and is improving the utilization. Also you can create your own widget. In my tip on MSSQLTips.com I’m showing how to create it.

I recommend install SQL Server Agent, Profiler, Reports and certainly whoisactive

sqlops06

How to install SQL Server SQLPS Powershell module

SQLPS module to Powershell helps to do so many things on SQL Server without SSMS and the main objective is to automate tasks for your environment.

This module isn’t installed by default, so there are a couple of steps before use it. First, install the module and the simple cmdlet install-module SQLPS won’t work.

To install the module you need install a couple of things (if you have SQL Server instance there is already installed).

The installation is quite simple and isn’t necessary to restart the system. There are 3 files necessary and need to install in the sequence below.

With all installed let’s try to import the module on powershell.

2018-08-01_20-36-01

Actually that didn’t work (yet). In the error message shows it’s not allowed to run scripts in my system.

Next step is to change the execution policy to run scripts in the system. Let’s try it using the cmdlets Get-ExecutionPolicy to know the actual value and Set-ExecutionPolicy to change it.  ( more information )

2018-08-01_20-45-23

Ok, second error and this time because I’m trying to change to RemoteSigned and only as administrator I can change it.

2018-08-01_20-54-28

Running as administrator the local machine execution policy has changed and the SQLPS module can be imported and use the cmdlets.

2018-08-01_21-00-16

Database migration methods

migration-logo-2389341When thinking about migration the most common issue is downtime. There are some methods that require downtime like backup/restore or less downtime using Log Shipping.

You can choose several different methods to migrate your database.  For example, methods that require downtime include:

  • Backup the database, manually copy the backup file to the machine and then restore the database there. This method is simple and the use of compression minimizes the time that is required.
  • Perform a backup to an Azure Blob storage account, and then restore the database from that backup. This method removes the necessity to manually copy the backup file.
  • Detach the database, copy the files to an Azure Blob storage and then attach them to the SQL Server instance. Use this method if you plan to store database files in Azure Blob storage permanently instead of on a hard disk.
  • Log Shipping, use a backup/restore process and the downtime depends on the transaction log file backup copying a small file in the process. This method is simple and is necessary a connection between the machines and permission to copy the backup files.

Methods that not require downtime:

  • AlwaysOn, adding a new secondary replica after replication has completed, you can failover to make the machine the primary replica. For Azure virtual machine use the Add Azure Replica Wizard.
  • Transactional Replication will minimize downtime, but don’t have an AlwaysOn deployment in your SQL Server system.

If you need help to decide which method is better for your scenario, feel free to contact me.

SqlPackage a tool to import/export SQL Server and Azure SQL DB

repair_database-512SqlPackage is a command-line utility that you can use for exporting and importing operations in both on-premises SQL Server databases and in cloud databases. SqlPackage supports the following operations:

  • Extract. Creates a database snapshot DACPAC file from a SQL Server database or from Azure SQL Database.
  • Publish. Updates the schema in a live database to match the schema in a DACPAC
    file. If the database does not exist on the destination server, the publish operation
    creates it.
  • Export. Exports both schema and data from a SQL Server database or from Azure SQL Database into a BACPAC file.
  • Import. Imports the schema and data from a BACPAC into a new database.
  • DeployReport. Creates an XML report that describes the changes that would be made by a publish operation.
  • DriftReport. Creates an XML report of the changes that have been made to a registered database.
  • Script. Creates a Transact-SQL script that you can use to update the schema of a target database to match the schema of a source database.
    Use the /Action: or /a parameter to specify which action to execute.

SqlPackage example to import from bacpac file to Azure
sqlpackage.exe /Action:Import /tsn:tcp:.database.windows.net,1433 /tdn: /tu: /tp: /sf: /p:DatabaseEdition=Premium /p:DatabaseServiceObjective=P4 /p:Storage=File

More about SqlPackage parameters:
https://msdn.microsoft.com/en-us/library/hh550080%28v=vs.103%29.aspx?f=255&MSPPError=-2147217396