Technical Talk – How our Smart Packages are tested before being deployed to endpoint clients

Is your organization seeking assurance that new application versions function flawlessly across hundreds or thousands of endpoints? Do you need reliable packaging and testing for new applications? Managing these tasks weekly can be overwhelming. Let us handle the testing, allowing you to focus on delivering value to your business.

Weekly updates and their testing

One major part of our services is maintaining customer environments and the applications deployed to the endpoint clients. Every week, we check which applications have received updates and update our smart packages accordingly. Before deploying these updated versions to our partners and their tenants, we conduct extensive testing to ensure that no faulty applications make it into production environments.
Our testing process is extensively automated, complemented by manual testing. An in-house tool checks all our applications to identify if new versions have been released since the previous week’s check. New versions are then automatically downloaded to a central location and tested to ensure the installers work correctly. Many issues are caught at this initial testing stage.

If the installers work correctly and don’t show any issues, they are packaged into our smart packages using another in-house tool, which streamlines the process. The updated smart packages are deployed to our testing tenant, where a large number of test machines, both physical and virtual, are waiting. Testing is also performed in several languages to verify applications like Firefox’s various locales.

During this phase of the testing process, we use another in-house tool, which simplifies verifying the versions and their functionality. This tool allows us to quickly check the status of the updated applications on each test machine. Logs sent to our analytics are also used. Applications are tested to ensure they launch correctly and do not exhibit any abnormal behavior. Most issues are identified at this stage since an application might install correctly but encounter issues upon launching after an update.

Applications that cause issues are further investigated and repackaged once the issue is resolved. Typically, these applications will be relaunched during the next weekly maintenance to undergo the entire testing process again.

Examples of Testing Process Successes

A while ago, certain CAD software changed their installer and the silent install parameters. The new installer installed .NET Desktop Runtime when installed from the GUI installer but not when installed silently. The application was repackaged to include this .NET runtime, which was run before it was installed or updated. This was a simple edit to the smart packages’ XML-based parameters. Without our testing process, this application might have caused issues on the user’s endpoint and prevented them from performing their tasks.

Another example is Azure Information Protection Viewer (now Microsoft Purview Information Protection). In an update earlier this year, the application stopped working on certain combinations of Windows language and region settings. While the application continued to work on many English-language Windows operating systems, Finnish and Swedish clients had issues even launching the application. In this case, we could not fix the problem ourselves, but Microsoft released an updated version soon after. Thanks to our testing, this faulty version was not deployed to production environments.

New smart packages and their testing

We have deep expertise in application packaging, and many partners rely on us for this service. We possess the knowledge, speed, and experience necessary for effective packaging. When packaging new applications or configurations, we conduct extensive testing to ensure they behave as intended. It helps when several people test a smart package to ensure nothing is missed before deployment.

The packaging process involves multiple stages of testing by different individuals. The initial packager tests the application’s installer for viruses or malware and verifies its functionality. All necessary parameters are thoroughly tested, especially if the smart package is partner/customer-specific. Once tested locally, the smart package is deployed via our Endpoint Management system to clients in Sandbox tenants. This step verifies that everything works correctly when the partner deploys via Intune. After the initial verification, the smart package is documented extensively and reviewed by another team member according to our testing process. Only after these steps is the smart package deployed to the partners’ tenants.
However, we cannot test everything. Some partner-specific packages require further testing by the partner, as we may not have access to their internal network or other necessary functions for the smart package to function properly.

Conclusion

Our thorough testing processes, both automated and manual, ensure that the applications deployed to endpoint clients function correctly and reliably. By handling these tasks, we help our partners focus on their core business activities, confident in the performance and stability of their software environments.