Home › Azure DevOps delivery forecasting
Azure DevOps
Delivery forecasting for Azure DevOps
Your backlog lives in Azure DevOps and someone wants a date. This page shows how to forecast from the work items your teams have already closed, with the odds attached, and how Lighthouse connects to Azure DevOps Services or an Azure DevOps Server inside your network.
What Azure DevOps offers on its own
The Forecast option on a backlog draws lines for the coming sprints from a velocity you type in. It sums Effort or Story Points, so it depends on how well the team estimates, and Microsoft's own documentation says it doesn't account for dependencies or blockers (Forecast your product backlog). The Velocity widget can count work items instead of points, and Analytics adds a cumulative flow diagram and cycle time charts with averages.
What you won't find in Microsoft's documentation is a forecast that gives you a date with its odds. For that you need a Monte Carlo simulation over your throughput.
Forecasting from closed work items
Throughput is the number of work items a team closes per day. Azure DevOps already knows it, because every item records when it reached its done state. A Monte Carlo simulation builds thousands of possible futures from those past days and counts how many of them make it. You get answers like "85% likely by this date" for a set of items, or "85% likely to finish at least this many" by a date. No estimates are involved.
Benji wrote about running these simulations against Azure DevOps with a nightly script back in 2023 (Running Monte Carlo simulations in Azure DevOps). The method in detail, with which date to promise and where it falls short, is in our guide to forecasting without story points. It applies to Azure DevOps just the same.
How Lighthouse connects to Azure DevOps
- Connection. You give Lighthouse your organisation URL and a personal access token. Paid editions can sign in with OAuth through a Microsoft Entra ID app registration instead, for Azure DevOps Services.
- Azure DevOps Server. Lighthouse runs on your own systems, so it can sit in the same network as an Azure DevOps Server and connect with a personal access token. Nothing has to be opened up to the internet.
- Teams. A team is a WIQL query, usually by area path, for example
[System.TeamProject] = "Shop" AND [System.AreaPath] = "Shop\Checkout". You choose which work item types count, such as User Story, Product Backlog Item and Bug, and map your states to to do, doing and done. A board wizard can read the query, types and states from an existing board for you. - Blocked work. Rules on any field, a tag for example, mark items as blocked. They feed the blocked and stale item views.
The work items Lighthouse reads are stored in its own database on your systems. The Azure DevOps setup guide in the documentation has the details.
Features and epics across teams
A portfolio is a second WIQL query for your features or epics. Lighthouse follows the parent links from team items to their feature and forecasts every feature in backlog order, continuously. When several teams work on one feature, it simulates each team and combines the results into one probability, which is never higher than the least likely team's, because all of them have to make it.
Features without child items yet get a size from a field you choose, or from your history, for example as many items as 85% of your past features needed. Lighthouse flags those features, so nobody mistakes an assumed size for a real one. Predecessor links show up as dependencies.
Forecasts back in Azure DevOps
On paid editions, Lighthouse can write forecasts into a field on the work item, for example the 85% completion date, or the item's age. People then see the forecast where they already look, without opening another tool, and Azure DevOps doesn't send notifications for those updates.
Other options for Azure DevOps
Lighthouse isn't the only way to get Monte Carlo forecasts into Azure DevOps. On the Visual Studio Marketplace you'll also find ActionableAgile Analytics for Azure DevOps Services, Agile Analytics by Baytek, Honest Cheetah by Benjamin Day Consulting, and FlowViz by Nicolas Brown, a Power BI template for flow metrics. Lighthouse is a separate application that you run yourself. That lets it bring teams on Jira, Linear or ServiceNow into the same portfolio forecast as your Azure DevOps teams. For ActionableAgile and Nave there is a side-by-side comparison.
Questions people ask
Can Azure DevOps forecast delivery without story points?
Not with its own Forecast tool, which works from a velocity in Effort or Story Points. You can forecast from throughput instead: count the work items each team closes per day and run a Monte Carlo simulation over that history. Lighthouse does this against Azure DevOps and gives you dates with their odds.
Does Lighthouse work with Azure DevOps Server on-premises?
Yes. Lighthouse connects to Azure DevOps Server with a personal access token. Because you run Lighthouse yourself, it can sit in the same network as the server, so nothing has to be reachable from the internet.
How do I tell Lighthouse which work items belong to a team?
With a WIQL query, usually by team project and area path, plus the work item types that count and how your states map to to do, doing and done. A board wizard can take all three from an existing board.
Can Lighthouse write the forecast back to Azure DevOps?
Yes, on paid editions. Lighthouse can write values such as the 85% completion date into a field on the work item, so people see the forecast in Azure DevOps.
What does it cost?
The Community edition is free for up to 3 teams and 1 portfolio, with Monte Carlo forecasts and flow metrics. A Premium license, sold as Self-Service or Enterprise, adds unlimited teams and portfolios, OAuth sign-in and writing forecasts back to Azure DevOps.
Try it on your own data
Lighthouse is free for up to 3 teams. Download it, connect your tracker and get a first forecast in about half an hour. You can try Premium for 30 days without signing up or giving us a credit card.