Updates to Approvals and Checks

[Microsoft]

Approvals and Checks provide increased security to your YAML pipelines. They allow you to control if a pipeline run is allowed to access a resource.

Let’s look at an example. Say you develop the FabrikamFiber web app. To deploy a new version of your app, you use a YAML pipeline that uses an Azure Resource Manager (ARM) service connection. Your deployment to production policy requires the app meet some performance criteria. You implemented the policy using an Invoke Azure Function check on the ARM service connection. Your Azure Function can check the performance of the to-be-deployed version of the system against a test environment and decide accordingly.

We are making changes to improve the scalability of Invoke Azure Function & REST API checks and customer experience.

Scalability improvements

We noticed that when an organization makes extensive use of the Invoke Azure Function & REST API checks, they do not scale. That is, a modest increase in the number of running checks leads to abnormally large delays in checks execution time, negatively impacting deployment experience.

Our solution to scale Invoke Azure Function and REST API checks is to enforce their configuration to match recommended usage. If you followed our guidelines, your checks are compliant and need no further changes.

Invoke Azure Function & REST API Checks can run in two modes:

• ApiResponse mode (or Synchronous mode): In this mode, Azure DevOps makes a REST HTTP call to your check and actively waits for an answer.
• Callback mode (or Asynchronous mode): In this mode, Azure DevOps still makes a REST HTTP call to your check, but does not wait for an answer. Once your check reaches a decision, it fires a callback to communicate it. Azure DevOps resumes your pipeline’s execution.

Two settings relevant for checks’ scalability are Time between evaluations and Timeout, both under Control options. Time between evaluations defines how long a check’s decision is valid. A value of 0 means the decision is final. A non-zero value means the system retries the check after the configured interval, when its decision is negative or there are other approvals and checks. Timeout defines how long Azure DevOps waits for a positive decision. Together, time between evaluations and timeout define how many times the system can try a check.

Your checks are considered compliant when they follow these rules depending on the mode you use and the number of retries:

• ApiResponse mode (or Synchronous mode): You limit the number of retries for your check to 10. You can do this in a number of ways. For example, you can configure timeout to 20 and time interval between evaluations to 2. Or, you can configure timeout to 100 and time interval between evaluations to 10. But, if you configure timeout to 100 and set the time interval between evaluations to 2, you are essentially asking Azure Pipelines to retry your check for a maximum of 50 times, and it is therefore not compliant. To summarize, the ratio of timeout to time interval between evaluations should be less than or equal to 10.
• Callback mode (or Asynchronous mode): In this case, Azure Pipelines does not retry your checks, and you are expected to provide a result asynchronously whenever you consider the result of your evaluation to be final. For this to be considered compliant, you must configure time interval between evaluations to 0.

There is no need to retry a Callback check. You fully control your check’s logic, so you know best if and when a pipeline run has met the criteria to proceed with its execution. There is no need for Azure DevOps to probe your Azure Function check every 15 minutes. All Azure DevOps needs is a decision within the configured Timeout. The decision must be final, be it fail or pass.

If your checks follow our recommended usage, they are compliant.

When Azure DevOps evaluates an Invoke Azure Function or REST API check, it makes an HTTP call. The check fails if the call does not complete in 3 seconds.

Rollout

Our goal is that by the end of the year, all your Invoke Azure Function & REST API check are compliant. You may need to take action to ensure this.

We realize this change is disruptive, so we have a three-phase rollout plan:

Phase 1

Starting June:

• When you create a check, its settings are compliant, but you can still edit them to make them non-compliant
• Azure DevOps shows warnings for non-compliant checks encouraging you to update their settings

We show warnings at resource level, for example, this is how it looks like for the FabrikamFiber Feed WUS1 environment, which has non-compliant checks configured.

When you open a resource’s details page, we show warnings on the Approvals and checks tab for non-compliant checks.

Finally, when you click on a check, we focus the input on the setting you need to update.

Phase 2

Starting July:

• You can only create compliant checks
• When you edit a check, you can save changes only if its settings are compliant
• Azure DevOps shows more prominent warnings for non-compliant checks encouraging you to update their settings

In addition to the warnings in Phase 1, we show warnings in a pipeline run’s detail page.

We show warnings in the Stages card. For example, this is how it looks for a stage named Deployment FabrikamFiber Feed WUS1 that accessed resources protected by non-compliant checks.

If you open the Approvals and checks side panel, we show warnings next to the non-compliant check.

Phase 3

Starting Autumn:

• Weekly, one-day long brownouts, where all pipeline runs that use non-compliant checks fail
• Starting Winter, all pipeline runs that evaluate non-compliant checks fail

A pipeline run that failed due to non-compliant checks shows an error in the Errors card guiding you to this blogpost.

Experience improvements

We are making reading check logs easier. Checks are critical sources of information for your deployment’s success. They can tell you if you forgot to close a work item ticket, or that you need to update a ticket in ServiceNow. Alas, knowing that a check provided such critical information is hard today. The pipeline run details page shows the latest check log only for checks that follow our recommended usage.

To prevent mistakenly approved Approvals, Azure DevOps shows the Instructions to approvers in the Approvals and checks side panel in a pipeline run’s detail page.

FAQ

How do I make sure my checks continue to work?

If you followed our guideline on recommended usage, your checks need no further changes.

Can I still use synchronous Invoke Azure Function or REST API checks?

Yes, but there is a 10-try limit.

My Azure Function takes more than 3 seconds to start up and process the request. What can I do?

If the HTTP call Azure DevOps makes to your check returns an HTTP status code outside of the 2xx and 3xx range, it retries for a maximum of 10 times. Then, the check fails. This is a built-in retry by the system that you do not have to configure anywhere.

I don’t see the check logs in my pipeline run details page. Why?

The pipeline run details page shows the latest check log only for checks that follow our recommended usage.

The post Updates to Approvals and Checks appeared first on Azure DevOps Blog.

View the full article