Let’s Compare: GP to BC Disaster Recovery
Most Dynamics GP environments have a good backup and disaster recovery plan or model, and it often consists of many layers. Many GP System Administrators or SQL database administrators know how to restore a GP company database and effectively ‘go to backup’ when the need arises.
But what happens when you move to the cloud, and start using Business Central? After all, you no longer have a server to log into to manage the database with BC, and you do not have SQL Server Management Studio (SSMS) to use for restoring a single database, or even simply querying the database tables.
This post is all about what happens when everything shits the bed, and you need to just ‘go to backup’ with Business Central, as compared to the same with Dynamics GP.
Noted - every GP and BC environment is different in some way, and every environment can’t rely on the same disaster recovery approach, tools, methods, or resources. However in its simplest form, a reasonable element of risk mitigation in the form of file and database backups is required for ERP system administrators to serve the going concern of the organization and keep the system online.
On “Going to Backup” (uh oh…)
The sooner the adverse situation is identified, the better.
Triage the problem - Tell the team to stop working, assess the situation thoroughly.
Take a clean backup of the current database or environment (do no harm!)
Target a past ‘point-in-time’ - The day, hour, minute, and second before when the catastrophic issue or change occurred.
Restore the entire database to that point in time (depending on method, sometimes this could also mean restoring the entire server instead). It is best to attempt the restore into a TEST company first and check it out, if possible, before doing the restore over a PROD company.
All transactions and changes after that point-in-time restore event are lost, and will need to be re-keyed, re-imported, re-exported, re-created entirely. Do this very THOUGHTFULLY, because some of these changes might have been EDI feeds, be transactions that require (re-)approval, or have actually been ACH payments to vendors that you do NOT want to send instructions for a 2nd time.
Remember, some ISV solutions may have their own database, or web sync, that needs to be coordinated after the restore of production ERP data in the event of a disaster (e.g. banking imports).
GP Databases
My best practice for maintaining GP environments (whenever possible) is to setup full nightly compressed backups of all of the SQL databases, and park them onto a separate “backup” partition on the machine. This way they don’t interfere with the other parts of the environment, and the entire drive itself can be further replicated to “offsite” cloud or network attached storage to extend the backward reach capability of the data files themselves. I also set the database recovery model to “FULL” on the production GP companies, and then configure an hourly transaction log backup to run in between each full backup.
I aim to keep up to 2 full weeks of continuous backups and transaction logs on the SQL server so that I can quickly restore any database within the environment to a point in time. This has truly ‘saved the day’ for many customers whom I have served over the years and truly been an effective means of rapid recovery.
GP Program Files
Since GP is a Windows-based application, it requires proper program files installation and configuration by an administrator. The best practice is to install GP on as few workstations as possible (ideally use an RDP server or RDWeb for granting access to end users). We also install the exact same products on all of the workstations so that the functionality and behavior on each machine is the same; if a product is missing on a machine (e.g. Fixed Assets), that functionality will not be present, and data entered from that machine could also be incomplete.
The core of the GP program files are the .exe, .set, .dic and .dll filetypes, among others. Additionally, GP requires specific permissions on certain folders on the workstation for temp and local files to work properly with the application. My best practice for preparing for disaster recovery on these files is to save a copy both locally, under a different name, and then also in my VersionControl folder, where I store all of the installation and service pack media for the current environment. I work with IT to ensure this folder is also part of the cloud or NAS retention policy along with the Backup drive; it would be needed in the event of a big disaster to reinstall Dynamics GP on a new server.
GP Reports and Forms
One of the most frequent areas where corruption occurs for GP comes from the Reports.dic file. This is typically a shared file among all GP users. Editing or modifying reports when other users are in the system, or attempted with improper rights, can introduce trouble. Keeping multiple file-level backups of the modified reports file, and individual reports (*.package), is commonplace for every GP System Administrator.
Business Central Databases
Be aware - a lot about the SQL databases for BC are different than for GP. It definitely takes quite some getting used to. When it comes to Disaster Recovery though, the System Administrator manages the BC environment fully in the cloud from the Admin Center. This console has one Production environment (which contains all of the companies in one container) and up to 3 Sandboxes (each is their own container). Controls for executing a disaster recovery situation are contained within each environment. Thankfully all of the Azure SQL environments for BC are automatically backed up and retained for 28 days. Additionally, deleted environments are retained and can be restored for 14 days (just in case!). Read more about this in the official documentation:
Certainly, restoring a PROD environment to a SANDBOX or a new PROD environment comes with some restrictions to be aware of. Specifically:
UNLIKE Dynamics GP, Business Central does not restore databases company-by-company. ALL companies are restored simultaneously. YIKES - something to get used to, especially if you have hundreds of them!
Any PTE customizations or extensions do NOT carry over to a sandbox and need to be redeployed by a developer.
Depending on how far back you need to go with your restore point, the latest versions of the application code will automatically be applied, and could be different than the data’s ‘point-in-time’ restore. You will see these emails start to come through after the restore completes and BC tries to catch up to the current release.
Finally, a number of services are stopped, configurations such as email are reset, and integration data is cleaned up to avoid problems.
For additional information about these and other restrictions and limitations, refer to the official documentation:
Restoring is a controlled process, performed in the Admin Center. In this example, you will see the steps it takes to restore an environment to a point in time in Business Central.
The restore of the BC database happens to its own separate environment (e.g. ProductionRESTORE), and using a different name than the original (e.g. Production). The original Production environment stays alive for a short while, so it can be used to export data for recreating in the restored environment.
You will want to turn off any job queues and revisit any scheduled tasks in both environments after restoring. Note that as part of the restore process, Advanced Options that handle a laundry list of environment cleanup tasks are available for after the restored database comes online.
This new Production environment changes the URL for everything, at least temporarily, while you check out the restored environment to ensure it has the pre-catastrophe state you are looking for.
Yes, this is one of those rare and extreme circumstances where you can ‘burst the bubble’ and have 2 Production environments at the same time, but only for 30 days. You might want to ensure user access to the old production environment is curtailed as well, so no new entries are made there which would need to be reconciled with the restored environment later.
While there is no doubt database size will impact the amount of time it takes to spin up the new instance to a point in time, so far my real-world experiences have been favorable and I was able to get them online within an hour of getting started, which is comparable to GP.
The final two major steps from a system admin standpoint are to DELETE the original Production environment that had the problem, and then RENAME the restored environment back to the original. This puts the proper/original URL back out there for users and Web Services and everything else connected to your BC environment.
Business Central Program Files
Since there is no Windows workstation, and BC is a cloud/browser-based application, there really are no local program files that can become corrupt like there are for GP. Furthermore, all of the code for all of the modules is already part of the codebase in BC, so onboarding new modules is often a matter of setting them up, testing, training, and making sure proper permissions are assigned to the users. Code updates typically come in 2 big waves each year, plus a number of smaller minor updates each month to fine-tune, optimize, and enhance functionality in the system, which is very unlike Dynamics GP program files / code changes.
Bottom line, the core program files are really not something a system admin needs to worry about for Business Central, outside of the posted release cadence. Customizations, extensions, or ISV solutions, may need to be updated periodically for compatibility and assurance purposes, but this is supposed to be done in a sandbox, in advance of the updates, and not part of a disaster recovery situation!
Business Central Reports and Forms
Again, BC has the distinct advantage over GP on report and form customizations. The report layouts are part of the database in BC, not a separate file like they are with GP. The BC reports are also therefore included in the automatic database backups, and they are much less susceptible to corruption than GP. I do recommend keeping local copies of the report layouts as singular backups, just in case. Forms (windows) in GP are known as Pages in BC. Extended pages are considered a customization, and will need to be redeployed to a BC sandbox after an automatic update, so coordinate with your partner or developer in advance.
Getting Prepared for the Transition to BC
One of the first things I learned when I crossed the bridge from GP to BC was how to perform disaster recovery, because it is clearly one of those things you can’t lose sight of when making the transition. The capabilities have improved quite a bit over the past few years as the platform and interconnectedness to other systems has grown.
Going to backup is an extreme circumstance, but it can be done pretty swiftly. So how do you prepare for the changes compared to GP?
Know the ramifications of going to backup. Sometimes data issues can be fixed with configuration packages or the Edit in Excel functionality. Custom apps can also be created that perform a task that corrects an issue.
Practice corrections in a Sandbox.
Make sure your BC System Admin knows the steps to do restore, practices it and is familiar with the process at least using a Sandbox environment.
Ensure your BC team has a communication plan. If something were to go terribly wrong or off the rails in BC, say something to your internal subject matter experts or support team.
Halt working in the application, get users logged out, and perhaps even cease integrations and pause job queues while you sort out the issue and devise a solution (don’t make the problem worse).
In a real event, proceed with the disaster recovery process ONLY after appropriate approval has been granted.
Microsoft Datacenters - Failover, Redundancy, much more
One of the things I geek out on as a Microsoft Partner are the sessions at conferences, and even online, that discusses and reports on the actual uptime and infrastructure behind the scenes. As a GP guy, I’ve watched servers evolve dramatically over the past couple of decades, but what I’ve seen in these sessions about the continuous investment, security, energy supply and consumption, load balancing, troubleshooting, communication across the many services teams, uptime and redundancy - I STAND IN AWE at what Microsoft has built in the cloud for Dynamics users. Check out a few of my go-to reference articles for discussing this topic:
Conclusion
The GP System Administrator has a lot of their work cut out in the future because so much is truly managed by Microsoft. A lot of worry about systems, licensing, installation, networking, disk space, backups, air conditioning, new hardware, etc. can now go away. I fully believe a savvy GP system administrator can move into a new type of administrator role managing BC, using the modern technologies, reporting tools, and connections across the entire Microsoft platform.
There is a LOT of material to study up on, and conferences to attend and learn more up-close and personal, but I hope this post helps clarify some of what you are up against in the event of needing to “go-to-backup” with Business Central!
I’m grateful should this experience or these insights be helpful to you on your journey…until next post!