Azure DevOps Dashboard

This dashboard uses SquaredUp’s WebAPI tile and data from Azure DevOps to give our DevOps Team the performance overview of our build and release pipelines that they always wanted.

Tim Wheeler, SquaredUp

How do I get it?

Challenge

While Azure DevOps does offer a dashboard, it is limited in the options users are allowed to query – e.g. each tile on the native dashboard must be set for a specific pipeline. Our DevOps Team needed a solution that would easily display data on our builds, release pipelines and agent usage.

Because we use many release pipeline stages across several different pipeline types, we needed a way to aggregate metrics across them. We also wanted to make the automation tests in our pipelines as efficient as possible, which meant we needed to gather metrics on problem tests and the length of time tests take to execute. Finally, we wanted to spend as little as possible on our pool of MS hosted agents – and to do this we needed to see how often all the agents in the pool are being utilized.

Solution

Using the WebAPI tile alone, we are now able to easily query the data we need from Azure DevOps with a dashboard. As the Azure DevOps results data can be accessed via https://analytics.dev.azure.com, all we have to do is use the Azure Analytics API to pull the data on our builds, release pipelines and agent usage – all into a single Azure DevOps metrics dashboard.

Dashboard walk-through

Build – a simple summary of the number of failed builds in the last 14 days.
Build Duration – the average time taken for our builds to complete. You may want multiple instances of this tile if you have builds that take different lengths of time to complete. Ours are fairly similar so it makes more sense for us to have a single tile to identify spikes in duration.
Build Failures – the number of failures in any release pipeline per day.
Build failure by stage name – a count of fails per release stage for the Azure DevOps Release pipeline.
Job queue – a count of the number of requests per day that end up in a queue.
MS Agent Usage – the total number of MS hosted agents that are in use on a given day. Our max number is 12, so if we see that number hit on multiple days, then we may want to increase the total number of agents in our pool.
E2ETesting failures – three tiles showing the count of fails for specific automation tests in the corresponding release pipeline stage.
Average E2E Testing Time – three tiles displaying the average run time for our automation stages.

How do I import and configure this dashboard?

Download pack for SCOM Dashboard Server

Ensure you're using SquaredUp DS v5.4 or above.
Already a SquaredUp customer?
Get the latest version of SquaredUp DS for SCOM
New to SquaredUp?
Try it for free
Create a Web API provider to connect with your Azure DevOps environment.
Use the following settings: [@portabletext/react] Unknown block type "tableBlock", specify a component for it in the `components.types` prop
1. In SquaredUp DS navigate to the right-hand menu ☰ > system > Integrations
2. Under Integrations click on Web API and choose either Simple, Basic Auth or OAuth depending on the API you are using.
  Note: When adding an OAuth provider you may get an message that the provider is not authorized if some of the provider configuration is incorrect (for example the username, password, secret etc), or if you use a proxy it could be that the proxy is not configured. See How to configure SquaredUp DS to use a proxy.
3. Complete the provider fields, using your API provider's online documentation as a guide.
  ignore invalid ssl: Turn the on/off switch to on if you are using a self-signed certificate.
  default headers headers sent with all requests using this provider, for example the API token in the format the API requires.
  url parameters url parameters sent with all requests using this provider. For example, a query parameter that is always used can be added here in the provider to save adding it to every dashboard configuration.
4. You are now ready to use this provider in a Web API tile (see How to use the Web API tile).
Download and import this dashboard pack.
Download pack for SCOM Dashboard Server
1. Download the dashboard pack zip file for the dashboard pack you wish to import.
  There may be additional steps before or after the import to get the dashboard working in your environment.
2. In DS for SCOM go to the top right hand menu ☰ > Import Dashboard Packs and drag the zip file into the Manual Import box.
3. The dashboard pack is imported and if the pack contains top-level dashboards, these will automatically appear in the navigation bar, usually in a folder called 'Community Dashboards' for dashboard packs from the SquaredUp Community. If the dashboard pack also contains perspectives (see Working with perspectives), then you'll see these when you drilldown to the relevant object.
4. Carry out any additional steps required to make the dashboard work in your environment. These steps will be described on the download page for each dashboard. For example, you may need to create the correctly named Web API provider, create a PowerShell profile, or edit tile collections.
5. Edit the imported dashboard as required.
Change the IDs to the IDs for you Azure environment.
All tiles on this dashboard except the Job queue and the MS agent usage tile require an Azure ID to collect data. Check each tile and replace the ID with the respective ID of your Azure environment.
1. Edit the tiles by clicking the Edit button.
2. Open the http mode panel.
3. Exchange the IDs with the IDs of your Azure DevOps environment.
Publish the dashboard.
A newly created dashboard will not be visible to others until it is published.
Only admins can publish dashboards, unless you have been given author permissions to a Team Folder see Team Folders
If you made changes to an existing dashboard, the changes will only be visible to others after you published the dashboard again.
You can identify a dashboard that has not been published yet or has unpublished changes by the unpublished button at the top:
When you click on the unpublished button, you'll have two options:
- Publish will make the dashboard or changes visible to others.
  Note: A newly created dashboard will appear on the menu where you created it. To move the dashboard to a different place on the navigation bar see How to edit the Navigation Bar.
- Discard will delete your draft dashboard if it has never been published or, if you made changes to an already published dashboard, discard the changes.
Publishing dashboards to different audiences
Find out how to publish dashboards to a subset of users using Team Folders or visible to anyone even unlicensed users with Open Access (Sharing Dashboards with anyone - Open Access).

GET STARTED TODAY

Zero to dashboard hero in 60 seconds

Start now with hundreds of customizable dashboards.