Migrate/Replicate Photos from Office 365 to Active Directory

Users can easily set their photos in Office 365 via any number of interfaces. Performing this change in AD is a little more challenging. Fortunately, we can quite easily write these photos back into Active Directory with a little PowerShell script. You’ll need the ActiveDirectory module and Exchange Online credentials with any multi-factor disabled as Exchange Web Services is used, which does not presently support MFA.

Encrypt Messages with Office 365 Message Encryption

There are a whole bunch of Encryption technologies in Office 365. Office 365 Message Encryption (OME) is Office 365’s way of providing end-to-end message encryption, without requiring any software beyond a web browser for the receiving party. It also does not require the recipient to be running Windows, or using Outlook, or for them to be an Office 365 customer. It is easy to automate, has no certificate requirement, and allows the recipient to reply in an encrypted fashion.

Automatic Message Encryption

Exchange Server administrators will be familiar with the concept of Transport Rules, now Mail Flow Rules in Exchange Online/Office 365. We can use Mail Flow Rules to kick off encryption when a particular condition is met. Below, we’ll explore a few common scenarios.

Before you complete any of the commands below, you’ll need to Connect PowerShell to Exchange Online.

Encrypt Based on Sender’s Group Membership

Say, for example, you need to encrypt all messages sent by the security team. Provided they’re all in a distribution group called ‘Security’ we can handle this like so:

Encrypt Based on Recipient Domain

If you deal with an organisation that requires all correspondence to be encrypted, this may come in handy. You can create a rule to encrypt all messages to a particular SMTP domain as follows:

Encrypt Based on Message Sensitivity (Private/Confidential)

This solution is a mechanism that allows for easy and transparent, but selective, rollout of OME behaviour. You can advise staff that marking a message as “Private” or “Confidential” in Outlook will cause it to be encrypted. This rule looks like the following:

Encrypt Based on Subject

This solution encrypts messages based on the content of a subject. This means that users can enter a specific string in a message subject to have it encrypted upon send. This might be useful if you already have plugins that categorize messages (think [SEC=UNCLASSIFIED]).

Thanks to this thread for prompting me to have a think about this one.

Cmdlets Used

Convert Excel Sheet to Json with PowerShell

This script converts a sheet from an Excel Workbook to a JSON file.


This is useful as JSON files allow all sorts of things that would break CSV files, like commas.
This would not be a pleasant CSV file to deal with:

DC01,,,Windows Server 2012 R2

This would look something like the following if parsed as a CSV file:

Hostname: DC01
?: Windows Server 2012 R2

Not ideal. JSON will display it like this:

"Hostname": "DC01",
"IPAddress": ",",
"OperatingSystem: "Windows Server 2012 R2"

It also allows for much more complexity than this, making JSON a great format for working with complicated sets of data. This script barely scratches the surface.


  • Windows PowerShell (v5.1 tested, please let me know what versions you’ve got it working on!)
  • Microsoft Excel (2016 tested, as with PowerShell, please let me know what versions you’ve got it working on!)


If you provide no input beyond the name of a spreadsheet, the script will confirm only sheet exists, and output the JSON file into the current directory:

Convert-ExcelSheetToJson -InputFile MyExcelWorkbook.xlsx

You can use the -SheetName parameter to specify a sheet name to export:

Convert-ExcelSheetToJson -InputFile MyExcelWorkbook.xlsx -SheetName Sheet1

Or, you can specify the output file as well:

Convert-ExcelSheetToJson -InputFile MyExcelWorkbook.xlsx -OutputFileName MyConvertedFile.json -SheetName Sheet2

The script also accepts an input file from pipeline:

Get-Item MyExcelWorkbook.xlsx | Convert-ExcelSheetToJson -OutputFileName MyConvertedFile.json -SheetName Sheet2

Where to get it

You can find the script on:

How to Upgrade AD FS 3 to AD FS on Windows Server 2016

Upgrading from AD FS on Windows Server 2012 R2 (AD FS 3) is a relatively straightforward procedure, which can be completed easily using the AD FS installation and configuration wizards.

Note: This information is compiled based on a preview build of Windows Server 2016 (Technical Preview 5). All information is subject to change in the public release of Windows Server 2016.

The Environment

My environment, prior to any upgrade, looks like this:

  • Active Directory domain corp.margiestravel.com. One domain controller, MT-DC1.corp.margiestravel.com
  • Two AD FS servers, MT-FS01.corp.margiestravel.com and MT-FS02.corp.margiestravel.com
  • AD FS servers load balanced by a Citrix NetScaler, though in theory, this should work with any other load balancing technology (F5, NLB, HAProxy)
  • One Web Application Proxy, MT-WAP01. Not domain joined
  • AD FS is published as https://sts.margiestravel.com/

After upgrade, the following will be introduced:

  • One AD FS server (Windows Server 2016), MT-FS16-01.corp.margiestravel.com
  • One Web Application Proxy (Windows Server 2016), MT-WAP16-01. Not domain joined

Blog post - AD FS upgrade

The Process

Put simply, the upgrade procedure involves installing new Windows Server 2016 servers, installing the AD FS role, then adding them to the existing farm. One of the Windows Server 2016 servers should be enabled as the primary server. Once the servers are in the farm and services redirected to them, the old servers can be removed.

Step 1: Install the new AD FS Servers

I’ll assume you’re capable of installing Windows Server 2016, and capable of installing the AD FS role on the server. We’ll pick up at the “you must configure AD FS step”.

  1. Install the Service Communications certificate (this should match that on your other federation servers)

    Import-PfxCertificate -FilePath C:\certs\mycert.pfx -CertStoreLocation 'cert:\localmachine\my' -Password $(Read-Host "PFX File Password -AsSecureString)
  2. Run the AD FS Configuration Wizard. Ensure Add a federation server to a federation server farm option is selected. Click Next
  3. Choose an administrative user to run the wizard as. If you’re a domain administrator, you can just hit Next
  4. Enter the FQDN of the primary AD FS computer in the farm
  5. Select the certificate you installed earlier
  6. Either choose the gMSA in use, or enter the service account’s credentials
  7. Review the configuration settings and hit Next. AD FS will configure on the server

Congratulations! Your AD FS farm now has a Windows Server 2016 server that can answer federation requests.

Step 2: Enable the new server(s) to receive requests

You can now shoot requests at these servers indiscriminately. AD FS on Windows Server 2016 behaves, from a user perspective, much the same way as AD FS on Windows Server 2012 R2, in this configuration.

Step 3: Promote a new server to be primary

AD FS has the concept of primary and secondary servers. Primary servers have a writable copy of the database, while secondary servers have a read-only replica of the database. Servers have no way of negotiating their role in the AD FS farm, so each computer must be updated manually:
Firstly, on the new Windows Server 2016 computer, set it to be the primary computer. This enables its database for changes, and means it will no longer sync its configuration from the previous primary computer.

Set-AdfsSyncProperties -Role 'PrimaryComputer'

On all other AD FS servers, run the following command to enable them to retrieve their configuration from the new primary computer. All servers must be updated, as they’ll be configured to point to the old primary server which will soon be merely a secondary server.
On the computer that was previously the primary computer, run the following:

Set-AdfsSyncProperties -PrimaryComputerName 'newprimarycomputername' -Role 'SecondaryComputer'

On other secondary computers, run the following:

Set-AdfsSyncProperties -PrimaryComputerName 'newprimarycomputername'


Step 4: Confirm functionality

Now that you’re running a Windows Server 2016 primary AD FS computer, you can start using some of the “cool new stuff” that comes with Windows Server 2016. Firstly, let’s use the new Get-AdfsFarmInformation cmdlet to make sure we can see our new server:

Get-AdfsFarmInformation | fl *
WARNING: PS0334: Farm nodes running versions prior to 'Windows Server 2016 Technical Preview 5' will not show in the farm node list.

CurrentFarmBehavior : 1
FarmNodes           : {MT-FS16-01.corp.margiestravel.com}


Step 5: Configure Web Application Proxies

There are two options to handle WAPs. WAPs can be replaced with a Windows Server 2016 server, or an existing Windows Server 2012 R2 WAP can be used.

Use existing Windows Server 2012 R2 WAPs

The great news here is that provided your Windows Server 2012 R2 servers can access the new AD FS server(s) by hostname, these will just keep on trucking

Deploy new Windows Server 2016 WAPs

If you want to go the whole hog and upgrade your WAPs to Windows Server 2016 at the same time, that’s pretty straightforward too.
First, build yourself a Windows Server 2016 server or two and get them on your network. Don’t domain join them. Install the certificate you wish to use for the WAPs (typically the same as the internal AD FS servers, but not absolutely necessary. The name must match, at least.

  1. Ensure that DNS resolution for your AD FS FQDN resolves to your internal AD FS load balancer. If it doesn’t, add an entry to the hosts file on each WAP: sts.margiestravel.com
  2. Install the Remote Access role with the WAP role feature:
    Install-WindowsFeature Web-Application-Proxy -IncludeManagementTools
  3. Make a note of the certificate thumbprint for your service communications certificate using the following PowerShell command:
    Get-ChildItem cert:\localmachine\my | where {$_.Subject -notlike "*ProxyTrust*"}
  4. Install the WAP into AD FS by running the following command on the WAPs:
    Install-WebApplicationProxy -CertificateThumbprint 2D125F4D735DEF823178E8F17E6579D04FB97B7A -
    FederationServiceName sts.margiestravel.com -FederationServiceTrustCredential $(Get-Credential)

You’ll now need to cut over your external load balancer or NAT rules to point incoming AD FS traffic to the newly configure WAPs.

Step 6: Decommission Windows Server 2012 R2 AD FS Servers

The good news is that your Windows Server 2012 R2 AD FS and WAP servers can now be removed – all functionality has been cut over to Windows Server 2016. How hard is that? Easy. Just uninstall AD FS or the Remote Access role from the servers.

Step 7: Raise the Farm Functional Level

Until now, your AD FS farm has been running at the Windows Server 2012 R2 farm behavior level (FBL). Do not confuse this with AD DS functional levels – these are totally different! Now that we’ve rid our farm of Windows Server 2012 R2 federation servers, we can enable all the cool new Windows Server 2016 AD FS functionality. This is done with a single PowerShell command that raises the functional level from “1” (Windows Server 2012 R2) to “3” (Windows Server 2016).

There is one important prerequisite to this: You must have extended your Active Directory Schema to support Windows Server 2016. It is not supported to utilise a Windows Server 2016 FBL without first using the Windows Server 2016 Active Directory schema. This is not the same as upgrading your domain and functional level, this is just adding the extra attributes used by Windows Server 2016 Domain Controllers. You can extend the schema manually (using ADPrep), or automatically by installing your first Windows Server 2016 domain controller.

To raise the AD FS FBL, open PowerShell as an Administrator and run the following command:


Hit ‘Y’ when prompted, and in a few minutes your farm will be running at functional level 3. We can verify this:

Get-AdfsProperties | Select CurrentFarmBehavior

How cool’s that? Open up the AD FS console and you’ll see all the cool new commands. You’ll also be able to run a bunch of new stuff in the ADFS PowerShell module such as:

PS C:\Windows\system32> Test-AdfsFarmJoin
cmdlet Test-AdfsFarmJoin at command pipeline position 1
Supply values for the following parameters:
GroupServiceAccountIdentifier: CORP\svc-adfs$
PrimaryComputerName: mt-fs16-01

Message                                                                        Context           Status
-------                                                                        -------           ------
Successfully verified the target computer is joined to a domain.               PreCheckTest     Success
group Managed Service Account was validated.                                   PrerequisiteTest Success
group Managed Service Account password information was successfully retrieved. DeploymentTask   Success
The Service SPN was successfully determined.                                   DeploymentTask   Success
The Farm Behavior Level was successfully determined.                           DeploymentTask   Success
Configuration was successfully retrieved from the primary server.              DeploymentTask   Success

Simple as that! You’ve installed a Windows Server 2016 federation server, migrated all the responsibility to it, and decommissioned your Windows Server 2012 R2 AD FS boxes.

What is ImmutableID in Azure AD?

Let’s talk about ImmutableID.

Immutable is a funny word. If you ask the dictionary, it’ll tell you that it means something like “unchanging over time” or “unchangeable”. If you ask Microsoft, prior to about 2014, they’d have agreed with these definitions.


Before we get started, let’s define a few things:

  • Active Directory Domain Services (AD). This is your on-premises directory service where objects are “mastered”. That is to say that the official “single source of truth” for anything we’re concerned about here is AD. You may have another product that feeds into AD, but we’ll treat whatever we see in AD as gospel
  • Azure Active Directory (AAD). This is the directory behind Office 365. Any object that exists in Office 365 (think user, group, contact, etc.) resides in AAD. It’s not exactly Active Directory, but it also kind of is. This isn’t really relevant, we just care that it holds all the information and behaves somewhat like active directory
  • AAD Connect, AADSync, DirSync. This is the magic glue. AAD Connect, formerly AADSync, formerly DirSync. AAD Connect speaks to both AD and AAD and works out what changes need to be made in each, if any. If an object is created in AD within the scope of AAD Connect, AAD Connect will create that object in AAD. If an object is updated in AD within the scope of AAD Connect, AAD Connect will update the object accordingly in AAD. There are a number of write-back scenarios available (password, device, group, etc.) but, for the most part, this synchronization is one-directional.

Understanding primary keys

In a relational database, primary keys are used to relate a record in one table with a record in one or more other tables. For instance, there may be a table called Buildings that has a record for each building within a company, then another table called MeetingRooms.

The Buildings table has a bunch of rows that look like this:

BuildingID BuildingName
1 Main Building
2 Old Building

The MeetingRooms table looks like this:

RoomID RoomName BuildingID
1 Party Room 1
2 Celebration Room 2
3 Kitchen 2

As you can see, based on the BuildingID value attached to each room specifics which building the room exists in. This allows for easy modifications to building attributes without having to update every single room (for example, if a building was renamed). As the ID fields here are referred to in multiple places, they must be configured as not only unique, but also immutable, or read-only once set. This means that if some bright spark tries to change the “Old Building” ID to 300, for example, all previous rooms aren’t orphaned (pointing to a building record that no longer exists, because the ID they refer to no longer exists in the database).

Unique Identifiers in Active Directory

While Active Directory has no “primary key” field like we’ve demonstrated above, it has a number of fields with values that are expected to never change. The one most are familiar with is the Security Identifier, or SID. This is a hexadecimal attribute that is displayed looking something like S-1-5-21-12345-1234-1234-500. SIDs are unique to the Active Directory forest, and are assigned only to user and group objects. Contacts and other object types do not receive a SID. SIDs are also subject to change when a user is migrated from one Active Directory domain to another within a forest.
Another attribute that can be utilised to uniquely identify an Active Directory object is the GUID. GUIDs are used extensively in the software development world to provide, in theory, a value that is globally unique across all systems and platforms. That is to say that, in all Active Directory forests around the globe, there should be no instance in which two objects have the same GUID. The same cannot be said for SIDs.
For this reason, objectGuid is commonly used in modern applications to uniquely identify Active Directory objects. The objectGuid remains with the user for as long as it lives in a particular Active Directory forest.

AAD Connect in a greenfield world

Let’s pretend, for a moment, that you’ve started a company called Contoso. You’ve set up your contoso.com Active Directory, and have decided that you want to use SharePoint Online within Office 365 for your intranet. Good work, you’re partway there. Now you have an Office 365 tenant with a bunch of licences waiting for users, and a bunch of users in AD waiting to collaborate.
To get the users from AD into AAD, we need to deploy AAD Connect. AAD Connect is installed on a dedicated server (not a domain controller, not an AD FS server, not a file server, not a print server, an AAD Connect server), and the configuration wizard run, accepting defaults for everything.
The relevant settings from the wizard look vaguely like this:

  • Synchronise all users, groups, and contacts from the root of contoso.com and below
  • Use objectGuid attribute to uniquely identify users

The following workflow now takes place for every single user, contact, and group object in Active Directory:

  1. AAD Connect finds user with User Principal Name areid@contoso.com (CONTOSO\areid)
  2. AAD Connect takes the user’s objectGuid and converts it to a Base64 string
    objectGuid: 7754d487-1fc3-4206-9e95-ce012f1586e5
    objectGuid.ToBase64(): h9RUd8MfBkKelc4BLxWG5Q==
  3. AAD Connect searches its copy of AAD’s information for an AAD user object with an immutableId value of h9RUd8MfBkKelc4BLxWG5Q==
    1. If a match is found the object is updated accordingly
    2. If a match is not found the object is created, stamping its immutableId field with the value determined above, tying the AD object to new the AAD object

There are a couple of additional scenarios that occur too:

  1. If a match is not found based on immutableID, but a user with a matching UPN/email address and a blank immutableId is found, the user will have its immutableId stamped with the new value (determined above) and the AD+AAD users become tied together. This is called a “soft match”
  2. If a match is not found based on immutableId, but a user with a matching UPN/email address is found but with a different immutableId, a sync error is thrown. AAD Connect will not override the user’s UPN, as they’re not the same object
  3. If a match is found based on immutableId, but the object types are different (e.g. AD user matching an AAD Connect contact), AAD Connect will throw an error and refuse to synchronize the objects. This won’t happen in the wild unless someone unintentionally (or intentionally) breaks something

The problem

With an understanding of how users are created and matched in AAD, and between AD and AAD, it is hopefully now becoming apparent that in the event of a user being moved from one forest to another, or deleted and recreated with the same details, AAD Connect will treat the new user as non-existent, and will commence the decommission process if the old “master” object is missing. Not necessarily ideal.
Before about mid-2014, there was no way to resolve this. If a user had to be moved from one forest to another, their Azure AD object, and as a result mailbox and other associated objects had to be recreated. Fortunately, now, Microsoft have provided functionality to resolve it, by allowing administrators to both clear and write to the immutableId attribute. There’s a couple of approaches to this, that I call the “safe way” and the “not-so-safe way”. I typically take the safe approach, as it’s much lower risk than the not-so-safe way.
NB: Both of these steps require a connection to Azure AD PowerShell, so make sure you’ve installed it and connected with Connect-MsolService before trying any of the below.

The not-so-safe way

As the safe way is only a few more steps on top of the not-so-safe way, we’ll start with this one. Put simply, we’re going to wipe the immutableId attribute so that AAD thinks this user has never been synchronised to an on-premises “master” object before. Basically this means that any AAD Connect sync will hit step A above (no match found, immutableId blank) and soft match based on UPN/email address.
Here’s the command to do this, you can run it for one user at a time, or for a few users, or for all users:

C:\> # For one user
C:\> Get-MsolUser -UserPrincipalName areid@contoso.com | Set-MsolUser -ImmutableId $null
C:\> # for a few users (in users.txt, one UPN per line)
C:\> Get-Content users.txt | % { Get-MsolUser -UserPrincipalName $_ | Set-MsolUser -ImmutableId $null }
C:\> # For all users (probably not recommended)
C:\> Get-MsolUser -All | Set-MsolUser -ImmutableId $null

This will clear immutableId and make AAD think these users are “In Cloud” principles, rather than synchronised principles, which means it’ll happily soft match them against AD as soon as it can.

The safe way

While slightly more involved to get going, “the safe way” is my recommended approach. Rather than taking a scattergun to your whole environment, this allows for forceful attribution of one object to another (the astute will notice how this mechanism also allows matching users that have absolutely nothing in common, which is useful rarely).
This approach takes a little bit longer than the last method, and a smidge more PowerShell-fu, but it is well and truly worth the extra effort, in my opinion. Since we can calculate what an immutableId should look like, there’s nothing stopping us from simply stamping an AAD object with the expected immutableId, making AAD Connect figure it’s been syncing those users all along. This approach looks like this:

  1. Import the AD module to allow retrieval of AD attributes
    C:\> Import-Module Active Directory
  2. Grab the GUID of our test user and transform it into an immutableId
    C:\> $userUPN = "areid@contoso.com"
    C:\> $guid = [guid]((Get-ADUser -LdapFilter "(userPrincipalName=$userUPN)").objectGuid)
    C:\> $immutableId = [System.Convert]::ToBase64String($guid.ToByteArray())
  3. Write that immutableId back to the AAD user
    C:\> Get-MsolUser -UserPrincipalName $userUPN | Set-MsolUser -ImmutableId $immutableId

If you’re feeling game and want to do this in bulk, the following should work too:

Get-Content users.txt | % { "Set-MsolUser -UserPrincipalName $_ -ImmutableId $([System.Convert]::ToBase64String((Get-AdUser -LdapFilter "(userPrincipalName=$_)")[0].objectGuid.ToByteArray())) " }

Whichever method you choose, hit go in your AAD Connect instance (or, more likely, wait for the sync task to run), and it will go and tie up all these users just like a bought one.

Further Reading