[NoBrainer] Azure Monitor Metric Alerts for Azure Functions

Last week, the Azure Function described in my last blog post, which acts as a webhook consumer/receiver, did not receive any events except heartbeat events. The heartbeat event is sent by the webhook every 10 minutes and showed me that at least the connection between sender and receiver as well as the middleware are working.

After an in-depth analysis, it turned out that something went wrong during the update of the webhook and the event filter for some reason changed from ["*"] (wildcard) to []. An update of the webhooks event filter resulted in the webhook delivering all expected events again. So far so good. But what if the webhook again does not deliver any events or only the heartbeat events? I asked myself what I could do to be alerted in such a case. As Azure Functions provide function execution count metric out of the box and because the webhook is expected to deliver at least one event besides the heartbeat events per hour, I decided to set up a Azure Monitor Metric Alert that gets triggered, if the Azure Function gets executed less than seven times per hour (5 to 6 heartbeat events expected per hour). On every alert, an email gets sent to the members of the Azure Resource Manager (ARM) role assigned to the corresponding action group.

The setup can either be done via the portal or as shown below using IaC (I did it with terraform in this case).

Versions:

• Terraform version: v1.4.7 (on windows_amd64)
• hashicorp/azurerm provider version: 3.77.0

Important note
To get emailed if an alert is triggered, the corresponding user needs to be assigned to role Monitoring Reader on subscription level!

Let’s have a look into the terraform file main-mon.tf.

# Monitoring
## Action Groups
resource "azurerm_monitor_action_group" "ag-3rd-level-support" {
  name                = format("%s-%s-ag-3rdLevelSupport", var.customer, var.env_code)
  resource_group_name = azurerm_resource_group.rg.name
  short_name          = "3rdLevel"

  arm_role_receiver {
    name                    = "MonitoringReader"
    role_id                 = "43d0d8ad-25c7-4714-9337-8ba259a9fe05" # id of built in role "Monitoring Reader"
    use_common_alert_schema = true
  }
}

## Metric Alerts
resource "azurerm_monitor_metric_alert" "ma-func-execution-count" {
  name                = format("%s-%s-ma-func-execution-count-per-hour", var.customer, var.env_code)
  resource_group_name = azurerm_resource_group.rg.name
  scopes = [
    azurerm_linux_function_app.func.id
  ]
  description = "Action will be triggered when Az function is executed less than 7 times per hour."

  criteria {
    metric_namespace = "Microsoft.Web/sites"
    metric_name      = "FunctionExecutionCount"
    aggregation      = "Total"
    operator         = "LessThan"
    threshold        = 7 # 5 to 6 heartbeat events expected per hour
  }

  action {
    action_group_id = azurerm_monitor_action_group.ag-3rd-level-support.id
  }

  frequency   = "PT30M" # alert rule gets executed every 30 minutes
  window_size = "PT1H" # alert rule considers last hour
  severity    = 1
}

resource "azurerm_monitor_metric_alert" "ma-func-http-server-errors" {
  name                = format("%s-%s-ma-func-http-server-errors", var.customer, var.env_code)
  resource_group_name = azurerm_resource_group.rg.name
  scopes = [
    azurerm_linux_function_app.func.id
  ]
  description = "Action will be triggered when HTTP server error occurs."

  criteria {
    metric_namespace = "Microsoft.Web/sites"
    metric_name      = "Http5xx"
    aggregation      = "Total"
    operator         = "GreaterThanOrEqual"
    threshold        = 1
  }

  action {
    action_group_id = azurerm_monitor_action_group.ag-3rd-level-support.id
  }

  frequency   = "PT1H"
  window_size = "PT6H"
  severity    = 1
}