Chapter 11 Monitoring

If you give developers and Azure users too much freedom, it can really quickly end in resources that are incorrectly configured, not aligned with the business, or just cost too much. The solution to this is Azure governance.

If all developers and system administrators just did what they thought best at the time, you could end up with a real mess of way more resources used than needed, the wrong VMs created, and so on. What is needed is governance of the process. Governance on Azure is a set of rules, policies, and roles to define acceptable use of Azure resources.

On Azure, it would restrict users from creating certain resources, what action they could do with existing resources, and any permissions for the Azure account in general. As you can tell from that, governance is crucial. Azure has several tools and services to help you implement adequate governance of your Azure resources.

The first is Azure Policy, which, as you may have guessed, is used to create policies in Azure.

Create Policies in Azure

Governance validates that your organization can achieve its goals through effective and efficient use of IT.

The first is Azure Policy, which, as you may have guessed, is used to create policies in Azure. As the first sentence for the Azure Policy documentation states, governance validates that your organization can achieve its goals through effective and efficient use of it. In other words, use Azure Policy to make sure users don't make a mess of it. A policy is a set of rules. Rules to make sure that standards and agreements within your corporation are followed and that resources are compliant with these policies. Look at it this way. If you have a bunch of Azure resources, you then also have a bunch of Azure policies defined for those resources.

Azure Policy, the service, is what ensures that the resources are complying with the policies. Azure Policy is your enforcer.

A policy is a set of rules to ensure resources are compliant

Define User Access

You can define specific user access to individual resources.

Minimum Access

RBAC can enable minimum access necessary toresources. This ensures only users with valid accesscan manage resources.

Target

Specific Use Cases Be very explicit about uses and access. For example, allow an application access to certain resources or allow a user to manage resources in a resource group.

The next part of Azure Policy is a fundamental part of how you use Azure. Role-based access control, often called RBAC, is a critical component in governance of users and their access to Azure resources. Role-based access control lets you define which users have access to specific Azure resources, what they can do with those resources, and what areas they have access to. One of the best practices for any computer infrastructure is to give users the minimum access they need. If a user doesn't have to access a database, well, then don't even give them access to it. It keeps users out of trouble, and RBAC can enable this. That means you can target specific use cases for assigning access. For example, allow an application access to only the resources it needs or allow a user access to all resources in a specific resource group. RBAC works through assigning roles to users, and a role assignment has three elements. A security principal, which is an object that represents what type of entity can get access to the Azure resource. This could be a user or group of users, for example. A role definition is a collection of permissions. A role definition lists the operations that can be performed, such as read, write, and delete. Scope is the set of resources that the access applies to. This is useful if, for example, you want a specific role assignment to have only access to a specific resource group. Role assignment is the process of combining those three properties to grant access to Azure resources. For example, let's say you have three virtual machines called admin, billing, and general. If you want to create role-based access to these resources, you could do it like this. An admin role. This has access to all three VMs with all permissions. An accountant role, which can access the billing and general VMs. It has read/write access to the billing VM and read access to the general VM. And, finally, a standard user role, and this has only read access to the general VM. The advantage is that you can now assign each role to any number of users. If there's a change to any of the roles, you only have to perform the change to the role, and all the users that have that role assigned will automatically get the updated role permissions.

A simple and efficient tool to manage changes and removal of resources are locks. You might want to ensure that a specific resource will not be changed or deleted, which is what locks are for. A lock can be assigned to a subscription, resource group, or resource level. A lock can be of type delete or read-only. Delete means you can't delete the resource, and read-only means you cannot make any changes to that resource. Once a lock is assigned to a resource, resource group, or subscription, it has to be removed completely before the actions are possible again for that resource. The first half of this lecture focused on helping with governance for users to enable them and also keep them out of trouble. We are about here now.

Assigning

Assign a lock to a subscription, resource group or resource.

Types

A lock can be of two types. Delete, where you can’t delete the locked object. Read-Only, where you can’t make any changes to the object.

Locked Means Locked

A lock needs to be removed before the locked actions can be performed again.

Blueprints are templates for creating Azure resources

The last part of the lecture on governance is looking at how you can make sure your Azure resources are both consistently created, deployed, and updated as well as secure. Now, remember, governance is here to keep you out of trouble. Now, the first tool in the second half of the lecture is Azure Blueprints. As you may have guessed from the name, blueprints are templates for creating Azure resources. It is a blueprint for everything you need to deploy for a standard cloud environment on Azure. Think of it this way. If you had to create a brand new Azure environment for a new product, and you had to meet certain governance rules and regulations, how would you do that manually? Most likely, you wouldn't, as it would drive you to insanity. Instead, Azure Blueprints pack everything you need, including templates for which resources to create user permissions using RBAC and any necessary policies. All in one easy pill - I mean, package. There are even built-in samples for the most common scenarios, including samples for scenarios with specific government regulations and guidelines. Similar to blueprints, but aimed at the organization that is considering moving to the cloud, is the Cloud Adoption Framework. This is a collection of documents that takes you through every step of the journey towards the cloud. You get guidance on how to define strategies for adoption, planning the move, what it means to be ready for the cloud, reasons for adopting the cloud, improving your governance and establishing practices around it, and managing a living, breathing cloud architecture. Well, the lot. Governance in particular is important to make a smooth transition to the cloud

Similar to blueprints, but aimed at the organization that is considering moving to the cloud, is the Cloud Adoption Framework. This is a collection of documents that takes you through every step of the journey towards the cloud. You get guidance on how to define strategies for adoption, planning the move, what it means to be ready for the cloud, reasons for adopting the cloud, improving your governance and establishing practices around it, and managing a living, breathing cloud architecture. Well, the lot. Governance in particular is important to make a smooth transition to the cloud

Collection of Documents

Lots of resources to guide you through the cloud adoption process.

Guidance

Help to define strategies for adoption, planning the move, “being ready” for the cloud, adoption reasons, governance practices, and managing a living, breathing cloud architecture.

Governance

Key to the cloud adoption process is governance of the process. The Cloud Adoption Framework is a big step in that process.

This is part of the Azure Security Center

Governance keeps you compliant and out of trouble.

• Azure Policy ensures that policies applied to resources are compliant.

• A policy is a set of rules to ensure compliant resources.

• Role-based access control (RBAC) ensures user compliance through assigning a role to a user. A role is a combination of security principal, role definition and scope.

• Locks make sure that subscriptions, resource groups or resources are either not modified or not deleted.

• Blueprints are templates for creating standard Azure environments.

• The Azure Advisor for Security Assistance is part of the Security Center.

Every service on Azure produces data as it performs its cloud duties. That data needs to go somewhere and preferably be used. Azure Monitor uses this telemetry to improve your Azure experience. Now managing cloud infrastructure very often means lots of services, lots of individual processes, and lots of balls in the air, but you still need to be on top of the health of the whole system and monitor everything that is happening. When things run smoothly, that's fine. Well, what about if some resource is not performing 100%, or another service has stopped altogether? This is the scenario Azure Monitor aims to solve.

Azure Monitor helps you find resources that aren’t performing 100%

Before we get that far though, let's cover what telemetry is. The definition of telemetry is: 'The collection of measurements or other data at remote or inaccessible points' and their automatic transmission to receiving equipment for monitoring. Yeah, that's from Wikipedia. Don't judge me. Translated into normal speak: telemetry is information about how services or devices are performing. This information is passed to a central point for further analysis. In Azure, the telemetry all goes into Azure Monitor, which gives you this lovely dashboard in the Azure Portal. The central point here is indeed Azure Monitor. For example, when virtual machines are running, Azure administrators need to make sure they're running smoothly the whole time. The second a VM isn't performing, you need to know. You don't need to know what data is on the VM, or what is this transmitting; you need the data about the machine, the telemetry. This telemetry is fed constantly into Azure Monitor that collates and monitors it all, and this happens with most Azure services. They will feed telemetry into Azure Monitor. Even services you host on premises can feed telemetry into Azure Monitor.

Constant Feed

Most Azure services feed telemetry data into Azure Monitor. Even on-premises services can send telemetry data to Azure Monitor.

Fully Managed

Azure Monitor is centralized and fully managed. You can analyze all the data from one place.

Query Language

Full access to an interactive query language to learn about the telemetry data.

Machine Learning

Predict and recognize problems faster with built-in machine learning.

Maximize Performance

Maximize Availability

Identify Issues

Monitor All the Things!

Azure includes multiple monitoring tools to gain full visibility into your Azure environment: Application Insights Azure Monitor

• Log Analytics

• Application Insights

• Azure Monitor Alerts

Azure Monitor generates A LOT of logs and telemetry data

Log Analytics stores and queries (or analyzes) that data to gain valuable insights

• Examples:

  • VM disk size

  • VPN connection logs

  • Long term analysis

  • Combine metrics for complex queries

"Now a deep dive in a KQL format is beyond the scope of this course in the AZ-900 exam. However, what you do need to know is that it is the querying language that is used to query different types of logs in Log Analytics."

"If you've used querying language such as SQL in the past, it is similar. However, this syntax is just a little bit different."

Performance insights for web applications

Answers questions:

• “How are users using our app?”

• “Where are our performance bottlenecks?”

• “Why are we getting website errors?”

Moving on, let's talk about our next monitoring tool called Application Insights. Again, the description is in the name and that Application Insights give you insights into your applications."

"Specifically, it provides performance insights for web-based applications.

Application Insights for your web-based applications is able to answer valuable questions such as how are our users using our web-based application? Where are different performance bottlenecks?"

"A few key points to be aware of when working with Application Insights is that, as we mentioned on the previous slide, this works with web-based applications only. Think webpages

When something breaks, alert someone to fix it!

Notifications in response to unexpected events

• Examples:

  • VM unresponsive

  • VM using excessive CPU

  • Application latency over 500 ms

With that said, it is available on a variety of different Azure and non-Azure services such as the Managed App Service service. Although you can also use it on Azure virtual machines that are hosting a website and you can even use it with non-Azure resources not hosted on Azure.

In this case, your virtual machines and non-Azure resources must have an agent installed which has been tied into the Application Insights service.

Typical Application Insight scenarios include the requirement to find performance bottlenecks in your web-based applications. Or if you want to have greater visibility into how users are using the website, maybe which pages are the most popular, Application Insights can fulfill that use case.

Dashboard

A personalized dashboard to highlight service issues affecting your resources.

Custom Alerts

Get notified of planned and non-planned outages. Alerts are simple to set up and customize.

Real-Time Tracking

Track any alerts and issues in real-time and get full reports once resolved.

Free Service

The Azure Service Health service is completely free.

"Whenever Azure plans on performing maintenance or there is a service incident, also called a whoopsie, it would be great if you were notified. Well, you will be with Azure Service Health. The Azure platform needs to be updated and maintained, just like any other computing infrastructure."

"While this is often done without you even noticing, it is important that you can mitigate the risk and take necessary steps to protect your infrastructure and applications. Downtime of your services and applications is the enemy. Downtime is evil."

"In Azure, there is the Service Health dashboard, right there in the Azure portal. Look at it. It's nice, isn't it? Azure Service Health notifies you about any planned and unplanned incidents on the platform. And the features include a personalized dashboard, that we just saw in the Azure portal, that shows the service issues that affect any of your resources."

"Custom alerts to notify you of any outages, planned or otherwise. These alerts are very simple to set up and part of the Azure platform by default. When an incident occurs, you can get the root cause of what happened, track the event in real time, and then download the official report afterwards. And it's a free service that should be one of the first things you configure for any new resources that you create."

"That is really it. So the next lecture is, well, it's right over here. Come on."

Azure knows about compliance and resources, and can give you recommendations through the Compliance Manager

Recommendations

Get recommendations for ensuring compliance with GDPR, ISO, NIST and others.

Tasks

Assign compliance tasks to team members and track progress.

Compliance Score

Chase a perfect score to be 100% compliant. Gamification for the win!

Secure Storage

Upload documents to prove compliance and store them securely.

Reports

Get reports of compliance data to provide to managers and auditors.

Compliance is not negotiable.

• GDPR, ISO and NIST are regulations and standards to ensure compliance with applicable legislation.

• Azure Compliance Manager provides recommendations, tasks to assign team members, a compliance score, secure document storage and reports.

• Azure Government Cloud provides dedicated datacenters to US Government bodies. Compliant with US federal, state and local requirements.

• Azure China region contains all data and datacenters within China. Complies with all applicable Chinese regulations.

Azure Information Protection

Classify, label and protect data based on data sensitivity.

Azure Policy

Define and enforce rules to ensure privacy and external regulations.

Guides

Use guides on Azure to respond and comply with GDPR privacy requests.

Compliance Manager

Make sure you are following privacy guidelines.

Privacy is an extension of compliance and just like any other online business, you need to take privacy seriously, which is why Azure takes it very seriously. In Azure, privacy is such a core part of the platform that there isn't a single service or place for it. Instead, the built-in privacy controls include tools and services that are also covered in this course in other chapters."

"Azure Information Protection is used to classify, label, and help protect data based on its sensitivity. Azure Policy, which we covered earlier in this chapter, is used to define and enforce rules to ensure privacy and external regulations. When dealing with GDPR privacy requests (that is, requests from European users of your data), use the guides on Azure to comply with these requests."

"Use the compliance manager to make sure you're following the guidelines around privacy, such as GDPR and ISO standards. All these tools are available to help simplify your privacy compliance. And, of course, Microsoft also has their own privacy statement. And because you chose to do this exam, you have to know about it. And that means I have to teach you as well. So thanks."

"Yeah. As with every other privacy statement on the internet, Microsoft explained how they're going to collect and treat your private data. Actually, it's so boring that I won't cover it here, but I've added the link in the resources for this lecture. So, enjoy. Enough about privacy. It exists. It's important. Make sure you know about it. Next up, you'll have to trust me.

Trust Center

Learn about Microsoft’s effort on security, privacy, GDPR, data location, compliance and more. A hub for more information about trust in each product and service.

Service Trust Portal

Review all the independent reports and audits performed on Microsoft’s products and services. Azure complies with more standards than any other cloud provider.

"What this entire chapter has built up to is obviously how much trust you have in Azure. Governance, Compliance, Privacy, and everything else are integral parts of trust in an IT system. There are two services on Azure that have to do with trust. The first one is the Trust Center. This is a shortcut to learn about all the things Microsoft does to make sure you don't lose trust in Azure and other Microsoft services. There are links to learn about security, privacy, GDPR, location of your data, compliance, and more. The links all take you to other Azure websites that can tell you more about the security implementation, privacy dedication, and so on."

"The other part is the Service Trust Portal. This is the location to review all the independent audit reports about Azure. It is their portal of proof that they're complying with a million different standards and certifications. It's crucial to know that Azure complies with all of the various quality and security standards. In fact, more than any other cloud provider. But you don't need to know which. That is what you can look up in the portal. If you want to read some audit reports on a Friday night, you know, just for laughs, the Service Trust Portal is a great hangout."

"The short of it is, Azure really wants you to trust their platform. Because without trust, there is no Azure. Coming up shortly is a trustworthy, brief, compliant summary."

Technical definition:

Centralized governance and management for on-premises and multi-cloud computing resources.

Simplified definition:

• Manage non-Azure resources as if they were in Azure.

• Extend Azure cloud management and services to non-Azure locations.

Governance

Governance on Azure uses several tools, including Azure Policy, role-based access control (RBAC), resource locks, and Azure Blueprints.

Azure Monitor

Collect telemetry data from resources, which you can then analyze. Maximize performance and availability and identify issues.

Monitoring Tools

Gain insights and receive alerts when something goes wrong with Log Analytics, Application Insights, and Azure Monitor alerts.

Azure Service Health

Notifies you about any planned and unplanned incidents on the Azure platform.

Compliance

Comply with GDPR and adhere to ISO and NIST standards. Use Compliance Manager to manage compliance. Use Azure Government as a US government body. Use the China region to comply with Chinese regulations.

Privacy

Core part of Azure and its products. Azure Information Protection, Azure Policy, and the GDPR guides on Azure are all privacy tools.

Trust

The Microsoft Trust Center is where you can find out what Microsoft does to keep your trust. The Service Trust Portal is where you can find audit reports and certificates awarded to Azure.

Azure Arc

Extend Azure’s control plane to your non-Azure resources. Apply Azure governance to Azure and non-Azure resources.