August 18, 2016 at 11:09 pm
Comments posted to this topic are about the item What's Your Smoke Test?
August 19, 2016 at 12:25 am
I have recently been involved with development and deployment of web services as part of a huge multi-system environment where the deployment process is a mixture of automated and manual deployments, generated reports and manual spreadsheets as well as a plethora of Word documents. Painful. They do smoke test various parts of the overall system but unless they automated that or used system monitoring systems then the coverage would be too low.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
August 19, 2016 at 7:04 am
So, in a smoke test what you should really be looking for is not functionality... but smoke.
- If deploying a DDL script, scroll through the output window and look for any errors or unusual warnings. Developers should code their T-SQL scripts in such a way that errors or warnings don't normally occur. For example, if the object already exists, then skip the create statement and don't confuse the DBA with alarming messages.
- If deploying a DML script, then perhaps the developer who coded the SQL has provided you with an exact or at least estimated number of rows that should be affected, and
you as the DBA need to confirm it's in line with the actual.
- Check the mssql logs or audit event logs for any errors that occurred during or after the deployment.
- Check for any resource and performance metrics are outside the normal range.
All of the above could be automated using T-SQL and/or PowerShell, but for a major deployment, show up and stick around in case users report an issue that needs your personal attention.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
August 19, 2016 at 10:23 am
It really depends on the deployment, depending on the components involved tests could be to run a complete system test to just making sure no errors were thrown during migration and crossing your fingers once users start hitting the system. This really requires whoever is doing the deployment to understand the components being migrated and the impact of testing various pieces of them.
Some examples of easy smoke testing are reports or file extracts, these can usually be migrated along with any DDL changes and run to completion with no end user impact.
On the other hand a change like how customer billing is processed likely can't be tested at all until it's usual run time.
Just because of how wildly those things can vary I've never been a fan of automated deployment or testing in production, a properly documented migration plan from the dev/test environments is much more effective.
August 19, 2016 at 11:22 am
Eric M Russell (8/19/2016)
So, in a smoke test what you should really be looking for is not functionality... but smoke.- If deploying a DDL script, scroll through the output window and look for any errors or unusual warnings. Developers should code their T-SQL scripts in such a way that errors or warnings don't normally occur. For example, if the object already exists, then skip the create statement and don't confuse the DBA with alarming messages.
- If deploying a DML script, then perhaps the developer who coded the SQL has provided you with an exact or at least estimated number of rows that should be affected, and
you as the DBA need to confirm it's in line with the actual.
- Check the mssql logs or audit event logs for any errors that occurred during or after the deployment.
- Check for any resource and performance metrics are outside the normal range.
All of the above could be automated using T-SQL and/or PowerShell, but for a major deployment, show up and stick around in case users report an issue that needs your personal attention.
some of these are more deployment verification. The smoke test is really to see if things are still working after this is done. Lacking errors in a deployment or a log isn't telling as to whether the system works. A smoke test for me is after deployment, does the GetCustomers method/proc still work? Can I log in with the app? Does the app server connect to the db?
August 19, 2016 at 1:13 pm
Steve Jones - SSC Editor (8/19/2016)
Eric M Russell (8/19/2016)
So, in a smoke test what you should really be looking for is not functionality... but smoke.- If deploying a DDL script, scroll through the output window and look for any errors or unusual warnings. Developers should code their T-SQL scripts in such a way that errors or warnings don't normally occur. For example, if the object already exists, then skip the create statement and don't confuse the DBA with alarming messages.
- If deploying a DML script, then perhaps the developer who coded the SQL has provided you with an exact or at least estimated number of rows that should be affected, and
you as the DBA need to confirm it's in line with the actual.
- Check the mssql logs or audit event logs for any errors that occurred during or after the deployment.
- Check for any resource and performance metrics are outside the normal range.
All of the above could be automated using T-SQL and/or PowerShell, but for a major deployment, show up and stick around in case users report an issue that needs your personal attention.
some of these are more deployment verification. The smoke test is really to see if things are still working after this is done. Lacking errors in a deployment or a log isn't telling as to whether the system works. A smoke test for me is after deployment, does the GetCustomers method/proc still work? Can I log in with the app? Does the app server connect to the db?
What I described above would be the backend smoke test from the DBA's perspective. This is done in conjunction with the user's front end smoke test. I've never even seen most of the applications that feed off of the databases that I support.
Does the GetCustomers method/proc still work?
That would be like a unit test that gets reproduced post deployment in production. Yes, I still do that type of thing for small incemental deployments.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
Viewing 6 posts - 1 through 5 (of 5 total)
You must be logged in to reply to this topic. Login to reply