But it works in my Environment - Tips for writing scripts someone else will run

TL:DR; Your scripts have too much in them and make too many assumptions, simplify them

One of the things I do in my day job is orchestrate procedures for a large enterprise. One that takes security seriously and so won’t give more rights than needed out.

A lot of third parties supply us with software and services and somehow we need to get these into the production systems, after first testing them in a reference system.

My team will not run a manual procedure as our experience is that even simple manual procedures often have an error in the documentation that if followed literally would probably result in it not working in the desired way leading to non-deployment or even worse destruction of data, be it a misspelling, steps out of sequence or an assumption about initial state. Whilst we could fix the procedure we’ll either get it wrong or end up in a protracted conversation with a frustrated developer. It was a 114 page manual installation procedure that finally changed our recommendations to requirements.

So how do you write a script that someone or something else can use without issues?

Well when we get an deployment package and it includes scripts we do a peer review, not for code style, definitely for security but also for usability. Following these guidelines we are able to significantly increase the number of successful first time uses.

I’m going to assume PowerShell for the examples but it equally applies to any scripting language, e.g. bash.

Don’t Do This

Continue

If you get an error, then stop immediately, don’t carry on because that error could be hidden in page upon page of output and may not be noticed.

$ErrorActionpreference = "stop"

Authentication

You don’t know how I’ll run the package you give me. You don’t even know if I’ll be the orchestrator

If its a true one off I may use the Azure Cloud Shell, PowerShell 7.1 at the moment, run under an MFA secured, named, just enough privilege, account. It will need an interactive login.

If its going to be done repeatedly, e.g. more than once, I will use an orchestrator, probably Azure DEVOps Pipelines but could be GitHub Actions, this will use a Service Principal or Managed Identity. It will need a certificate based login.

If its going to be done regularly I will use timer based orchestrator, probably Azure Functions. It will need a certificate based login.

If its going to be done in response to something I will use an event based orchestrator, again probably Azure Functions. It will need a certificate based login.

If it needs an account login and .NET, think Exchange scripting, I’ll run it from a Windows Virtual Machine. It will need an interactive login.

That’s two types of login but in practice we end up with others as well, client id and secret, app id and secret, SAS token, the list is quite long.

So your script should contain no authentication logic at all. The orchestrators will handle all of this and set the context right. It’s the same set of steps for multiple scripts and it changes over time. Most orchestrators have a library of authentication tasks that handle all of this.

So no code like:

Connect-PnPOnline -Url:"https://yoursystem.sharepoint.com/sites/deploymentsite"

Secrets

Following on from not doing authentication means you won’t be asking us for passwords to be typed in and stored in memory in plain text, or even worse output them into logs.

However there are other secrets, or certificates, that you may need and again as you don’t know how the script will be orchestrated you can’t know how they’ll be stored, could be in a book, in a password manager, in an Azure Key Vault, etc.

If you do need a secret that can’t be established from context then consider either of these two mechanisms either of which mean its up to the orchestrator to decide how to store and retrieve secrets.

Use a named environment variable

Use-PowerShellVerb -SecretParameter:$MySecret

Use a Secure String

[Parameter]
[SecureString]
$MySecret

Swap context

In a single script don’t swap context, e.g. if using PnP PowerShell don’t connect to multiple Sites, that would mean you have to include authentication, it also makes it very hard to understand the script.

Instead supply one script for each context and if you need to swap contexts also supply a script that orchestrates calling each of the other scripts. We’ll turn this into a suitable orchestration method.

Frankly this separation makes scripts easier to write and test anyway.

On rare occasions you will need multiple contexts in a single script, not for migration, that should use ETL but typically when dealing with streams, especially transform streams. The mechanisms that use these tend to have proscriptive parameter requirements so in your script assume that the stream is set up externally or that its reader and writer will be passed as parameters.

Clear your Output

This will affect what’s logged by the orchestrator, or may actually cause the script to fail in some rare cases.

So no

cls

It might make it easier for you not to have to type it but when I automate a script or an orchestrator uses it we set up a clean environment every time so that doesn’t affect us and if something goes wrong I want as much output as possible to send back to you. If it really bugs you write an orchestration wrapper that clears the screen before running your script.

This is particularly important if the orchestration requires multiple scripts to be used.

Assume Location or Operating System

You don’t know that I’ll use the same folder you do as my working folder, some orchestrators don’t allow it, some download to specific folders or use symbolic links.

Similarly cloud shell runs Linux not Windows so path’s use \ and not /.

FOr safety use \ in all cases and make all paths relative to the script being run.

Fall Asleep

This one seems to afflict Sharepoint developers a lot, they put 30 second sleeps between provisioning statements.

What a sleep statement tells me is you don’t know if the call your making is synchronous or asynchronous. Your assuming it asynchronous and N seconds will allow it to finish.

Don’t do this

Do-MyThing
Sleep 30

I could have written this as Do Be Deterministic instead.

If a call is synchronous, as most are, no need to sleep after it returns the task is finished.

If the call is asynchronous then use a suitable mechanism, for example call back, await or polling.

Comment Out Code

It makes me wonder if that call was just commented out whilst you were testing but is actually needed. It also tells me you don’t have, understand or trust your source code control system.

If you want it run put it in the script, if you don’t want it — remove it.

Don’t do this

# Invoke-IsNeeded -Probably

Do

Make it Idempotent

This is critical assume your script will fail through no fault of your own and when it does it can just be rerun without having to take any additional steps.

I cannot emphasise this enough, your scripts job is to ensure that the system ends up in the desired state no matter what its initial state.

How can a script fail if it has no faults? Multiple ways, lack of authorization, a resource it depends on being down, environment in needs temporarily unavailable, another process is running that affects it.

We received a deployment that was a complex set of virtual machines, networking, services, and storage accounts. We were running it when the had the fire at Azure UK South and lost the ability to provision virtual machines but because the supplier had supplied an idempotent deployment when UK South was fully available again we just ran the pipeline again and it completed successfully.

Document

It doesn’t have to be exhaustive but document what you expect the script to do. Where possible do this in the script itself so it doesn’t become detached or lost.

If there is more than can be placed in the script or there are components, such as orchestration or external configurations then use Markdown to document them. Why Markdown because your script will be stored in a Source Control Repository and markdown is text based.

Know Your Scripting Language

Make use of the features the scripting language gives you. PowerShell is especially good in this regard, with structured headers for modules and functions that can assist in your documentation see Functions — PowerShell | Microsoft Docs.

Constrain Your Parameters

Test that the parameters supplied are valid and if not stop immediately, again PowerShell has very good support for this, Functions — PowerShell | Microsoft Docs

Provide Test Mode

If at all possible provide a ‘test’ or ‘What If’ mechanism so I can orchestrate the process and see what the effects will be without actually applying them. I can supply these back to you or on to the project teeam to confirm its what they want without actually affecting production. Again PowerShell is well supported for this see Functions — PowerShell | Microsoft Docs

Final Thoughts

The suppliers we work with do tend to object to these ‘requirements’ and think the resulting scripts are ‘thin’ and the end results ‘too simple’ but once we do a successful deployment they do seem to adopt them.