Chapter 9 Azure Solutions

We'll start with one of the solutions in Azure that is becoming more and more prominent. A bunch of seemingly random devices connected to the internet to perform a larger task and to explain the Internet of Things, let's cross to my identical twin. Internet of things.

the Internet of Things is a system of interrelated computing devices, mechanical and digital machines, objects, animals, or people that are provided with unique identifiers and the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction. Now, in other words, pretty much anything you can imagine as long as human interaction isn't needed for the data to be collected and sent.

So, let's look at which IoT services Azure have to help you out. There are two main ones we will cover. The first one is Azure IoT Hub. As you can deduct from the name, it is a hub to collate data feeds from all your things. It is what is called a backend, meaning the system they receive the data and manage it. It can receive data from millions and according to Microsoft, even billions of devices. This is the power of IoT on Azure, as well. Because these devices are meant to be autonomous and do their tasks without any human interaction, you could have a lot of them, billions even.

• Managed and Secure

• Ease of Deployment

• Platform as a Service

• Scaling and Authentication

So, imagine a water network for a big city. The water comes from far, far away and having it running smoothly is critical to ensure people in the city can open the tap and get fresh water. This makes the water network critical. There will be thousands of valves, pumps, gates, locks, and much, much more. And if any of these things fail, it could be disastrous for the water supply to the city. If every single thing was connected to the internet, then if any of them failed, you could see it instantly. It would create a much more stable, and over time, much more cost-effective network. You wouldn't have to spend time and money checking the network constantly in the field. IoT hub on Azure will ensure that the entire IT solution is managed, secured, and that new devices can be deployed with ease. It is a Platform-as-a-Service offering or PaaS from Azure. You get managed infrastructure, scaled resources, and device authentication.

Software as a Service

Simplify and speed up the implementation of your IoT solution.

No Coding Needed

You don’t have to know how to write code to deploy your IoT project. Receive feeds from devices and focus on metrics and business value.

Pre-made Connectors

Use any of the hundreds of connectors that are ready to use in IoT Central.

The second Azure service for IoT we'll look at is IoT Central. This is the Application Platform-as-a-Service or aPaaS offering for IoT on Azure because we need more acronyms, hmm. IoT Central helps simplify the setup of your IoT solution and can be a really fast way to get your IoT solution running. You don't have to write any code to connect all your things. IoT Central receives the feeds from your devices and provides you with dashboards, metrics, and rules to manage it all. There are hundreds of ways to connect the IoT system and data to your other business systems, all through pre-made connections in Azure.

IoT Hub

PaaS solution that provides more control over the IoT data collection and processing.

IoT Central

SaaS solution that provides premade IoT connections and dashboards to get set up quickly.

That gives you two ways to start an IoT project. Use IoT Hub for solutions where you want more control of the whole process and how you collect and manage the data. IoT Hub is a PaaS solution and creates a secure link for your data connections to the devices and makes provisioning, that is getting them on the network and system, a lot easier to do for millions of devices. IoT Central is an aPaaS solution that provides all the dashboards and infrastructure you need to create an IoT solution without any cloud nor IoT knowledge. It is a tool with many ready-made components for you to set up.

An all-in-one solution for IoT devices on Azure.

Specific Hardware

You can only use hardware and chipsets certified by Microsoft for use on Azure.

Security

Specialized security service that manages maintenance, updates and general control.

Operating System

Custom made for Azure Sphere devices. Connects to the Sphere Security Service.

"But Lars, what about the devices themselves?" you might ask. Very good question. And of course, Azure has an answer. Azure Sphere is a kind of all-in-one solution for IoT devices. You have to use specific hardware chipsets approved and certified by Azure to use Azure Sphere, you get a specific Azure Sphere security service, which manages maintenance, updates, and control of the Sphere IoT devices. The Azure Sphere operating system is what runs on the device. This is a piece of software made specifically for the Sphere devices and will connect to the Azure Sphere security service for YouTube. So Sphere mandates what hardware you can use but then gives you security features and an IoT development platform in exchange. Neat, all right. Next, we look at how to manage very large amounts of data possibly collected from IoT devices."

Modern business system processes very often mean collecting data. Lots of data. Like, enormous amounts of data. So let's talk about big data. Data is everywhere we look: how many cars are passing by, what the temperature is, how many newspapers are sold, how many visitors to a museum, and on and on. That data can be incredibly valuable for businesses, and they collect tons of it. Now, the collection of data is not that hard. Store everything that passes through your system and - voila - you have data. However, getting any value from it at all is a very different matter. That's where the idea of big data comes in. How big is big, though? Well, it's big. Like, really big.

So big that most common software applications wouldn't be able to collect, store, or process it. Think data from millions of devices in an Internet of Things scenario, for example. Big data keeps changing, though. As we can process more and more data, the definition of what is big data moves as well. So back to the value from all of this data. Businesses want to get value from big data to provide better service, better products, and ultimately be more competitive and profitable. Azure, of course, has tools and services to help with this.

Data from Millions of Devices

- The definition of how big “Big Data” is changes as the industry can process more and more d

Big Data = Better service,

Better products,

More profits

Large Amounts of Data

A data lake is a very large body of data.

Parallel Processing

Two or more processes or computers processing the same data at the same time. Data Lake Analytics includes parallel processing.

Ready to Go

Servers, processes and any other needed services are ready to go from the start. Jump straight into the data analytics.

Azure Data Lake Analytics is a great visual name. You have a massive body of data (a lake) that you can perform analytical procedures on. The main thing to understand when processing enormous datasets is that you need parallel processing of that data. This is two or more computers or CPUs that process the same dataset at the same time. They do it in parallel. With Azure Data Lake Analytics, you can do this without having to manage servers, virtual machines, or clusters of computers. You can jump straight into processing the huge dataset with thousands of parallel processes. Another similar offering is Azure HDInsight. Yeah, I know the logo totally rocks. I mean, who doesn't like a yellow elephant? Now this service provides a very similar service to Azure Data Lake Analytics but uses all open-source frameworks.

⁃ Similar to Azure Data Lake Analytics

⁃ Open Source, which is free and community supported

⁃ Includes Apache Hadoop, Spark and Kafka

Open-source framework is another way of saying software systems that are free to use and for which the community of its users help update and maintain it. Azure and Microsoft in general support a lot of open-source projects, and this flows into products on Azure as well. HDInsights include products such as Apache Hadoop, which allows many computers to work together to solve problems that involve massive amounts of data; Apache Spark, which is an interface to allow parallel program between clusters of computers; and Apache Kafka, which allows processing of real-time data feeds.

⁃ Based on Apache Spark, a distributed cluster-computing framework.

⁃ Run and process a dataset on many computers simultaneously.

⁃ Databricks provides all the computing power.

⁃ Integrates with other Azure Storage services.

Azure Databricks. Yes, I'll admit. I also thought it was some huge beast of a computer to hold bricks of data, but no. It is a cloud service on Azure. Databricks is based on Apache Spark, which is an open-source distributed cluster computing framework. Got that? Yeah. It means that Apache Spark can run on many computers at the same time and also process the same set of data on multiple computers at the same time. Neat! The advantage of using Azure Databricks is, of course, that you don't need to buy lots of computers, nor maintain them. Azure provides the computing power and on top also offers a lot of integration points with all the other storage services on Azure, such as Storage, Analytics, and even Data Lake storage and Hadoop storage. You use data from all these data sources to find insights and analyze the data in Apache Spark or Azure Databricks, as it is called on Azure.

⁃ Azure’s data warehouse offering

⁃ Used to be Azure SQL Data Warehouse

⁃ Used for reporting and data analysis

⁃ Only limited by your scope

⁃ Use Synapse SQL language to manipulate the data

The last service you need to know about for the exam is Azure Synapse Analytics. This is the data warehouse solution for Azure. In fact, it used to be called Azure SQL Data Warehouse. A data warehouse is a system used for reporting and data analysis and is considered a core component of most data-driven business decisions. The strength of Synapse Analytics is that it is in the cloud. It is limitless in storage and analysis capabilities - or at least, well, only limited by your wallet, scope, and the size of Azure data centers. Azure provides the Synapse SQL language, which you can use to cut, dissect, split, combine, and in any way mash up the data in any way you need.

Speed

Speed and efficiency of processing large amounts of data, provides real value.

Cost Reduction

Save large amounts of money on storage and processing, by using a Big Data solution in the cloud.

Better Decision Making

Immediate data processing and analysis in-memory means you can make better decisions, and make them faster.

New Products and Services

Understand what customers want and provide them with much better products and services.

So what do you get out of all this data and work, though, that all these big data services do? Analytics of business data is not a new concept. However, the speed and efficiency which you get from big data on Azure is the real value. You can find trends and insights that you can act on straightaway. And this leads to three main outcomes. Cost reduction. Big data technologies, such as cloud-based analytics, bring significant cost advantages when it comes to storing large amounts of data. Faster, better decision making with the speed of in-memory analytics, combined with the ability to analyze new sources of data. Businesses are able to analyze information immediately. And, of course, new products and services. With the ability to gauge customer needs and satisfaction through analytics comes the power to give customers what they want. This means companies can create new products to meet customers' needs much more efficiently. Up next, the evil robot overlords are taking over. Well, sort of."

Chances are that you've heard stories--fictional and nonfictional--about machine learning as well as artificial intelligence. And of course, Azure has an offering for those too. Otherwise, I probably wouldn't include it in the course either. Hmm. Anyway, AI is the capability of machines to imitate intelligent human behavior. Through AI, machines can analyze images, comprehend speech, interact in natural ways, and make predictions using data. So that's Microsoft's definition. In other words, we are teaching machines to make decisions based on what they learn.

On the Azure platform, AI is most often called machine learning. Machine learning is a subcategory of AI, although they're often placed together in the same bucket. Microsoft focuses on 3 main parts to how machine learning can be used on their cloud platform.

1. Models

   The definition of what your machine learning application is learning. A model is a set of rules on how to use the data provided. The model finds patterns based on the rules.

2. Knowledge Mining

   Use Azure Cognitive Search to find existing insights in your data (e.g., file relationships, geography connections and more).

3. Built-in Apps

   Azure has a number of built-in apps you can use for machine learning and AI straight away. These include cognitive services and bot services.

On the Azure platform, AI is most often called machine learning. Machine learning is a subcategory of AI, although they're often placed together in the same bucket. Microsoft focuses on 3 main parts to how machine learning can be used on their cloud platform.

The task of creating models for teaching your AI application is the essence of machine learning. A model is the way that you define what you want your machine learning implementation to learn. You give it a model, which is a set of rules, and the application then starts playing this model over and over again with the data you've provided. Over time, usually very fast, the model will find patterns in the data that follow the rules you have provided.

Knowledge mining is when you ask Azure Search, a service from Azure, to find insights that are present in your data. This could be relationships between files, data based on geography, images with the same person in them, and much more.

Azure PaaS offering that lets you build bots for Q&A services, virtual assistants and more.

Code or Visual

Create a bot using the visual editor or programming.

Language

Let users talk to your bot with natural language and speech integration.

Integration

Integrate the bot with other services, like Facebook Messenger, Teams, Twilio and more.

Branding

Use your own branding and own the data the bot uses and produces.

Ready-made built-in apps on Azure, such as Cognitive Services for recognizing speech and language or Bot Services to create an automated process to answer questions in an interactive way. Bots are what you may have seen on websites as "Assistants, ready to answer your questions!" or something like that. The Azure Bot Service is a PaaS, Platform as a Service, service that you can use to build your own bot, and you can build any kind of bot from a simple question and answer bot that has a predefined list of questions and answers, surprise, surprise, to a complex virtual assistant bot for your customers. Lots of bots. The Bot Service has some cool benefits and features, too. You can create a bot using either the visual composer or by programming the code yourself. The visual composer is a non-code way of getting your bot into play quickly. Add natural language and speech easily to catch common inquiries from customers before they take up time from support staff. Integration channels, such as Facebook Messenger, Microsoft Teams, Twilio, and many, many more. Put your own touch on the bot using your brand and keep ownership of any data that goes into or comes out of the bot. A bot is a quick way to add an extra layer of customer service to your business or product.

1. Vision

   You can use the Vision service to recognize, identify and caption your videos and images. Automatically.

2. Decision

   Apps can make decisions based on content. Identify potential offensive language, detect IoT anomalies and leverage data analytics.

3. Speech

   Automatic speech-to-text transcription. Speaker identification and verification.

AI as a service on Azure comes in a few different forms. They're all part of Azure Cognitive Services, which is a collection of tools to use in your own applications. The Vision service, which provides information about visual content found in images and videos; it will identify visual objects, people and concepts, as well as caption the media for you too. The Decision service can make informed decisions based on data available to an app. This could include identifying potential offensive language, get notified of anomalies in your IoT network, or leveraging data analytics from your business. The speech service takes any spoken audio stream and then converts it into text as a transcript. The service can also identify unique voices and even verify a speaker, based on the speech.

• Supports all Azure Machine Learning tools.

• Pre-made modules for your project.

• Use for real-world scenarios, such as Twitter sentiment analysis, photo grouping and movie recommendations.

The main tool for using machine learning on Azure is the Machine Learning Studio. This is a visual tool that can help you manage all of your machine learning needs. It supports all the various programming languages, data sets, models, projects, and services that are available in Azure for machine learning. It has pre-made modules that you can just plug into your project and use straightaway. Some of the use cases for Azure Machine Learning Studio could be a Twitter sentiment analysis of all of your followers, grouping specific types of flowers in photos or recommending movies for users based on data about them. Just as important as the Machine Learning Studio is, so is the actual Azure Machine Learning Service.

This is the end-to-end service for using machine learning pretty much everywhere on Azure. It's a service that expands all of Azure and helps you use machine learning in more and better ways. It's a range of tools to help you build machine learning into your products and Azure processes. And you get automated machine learning, which means Azure will pick up trends and processes automatically for you and create machine learning models that you can use.

1. End-to-End Service

   Enables you to use AI and machine learning almost anywhere on Azure.

2. Tooling

   The Azure Machine Learning service is a collection of tools to help you build AI applications.

3. Automation

   Azure automatically recognizes trends in your applications and creates models for you.

. Just as important as the Machine Learning Studio is, so is the actual Azure Machine Learning Service. This is the end-to-end service for using machine learning pretty much everywhere on Azure. It's a service that expands all of Azure and helps you use machine learning in more and better ways. It's a range of tools to help you build machine learning into your products and Azure processes. And you get automated machine learning, which means Azure will pick up trends and processes automatically for you and create machine learning models that you can use.

So AI is about better products and better customer service, much more than taking over the world. Next up, the world without virtual machines, hmmm."

Yes, there are servers You just don’t manage them it is extreme PaaS

Managing virtual machines, infrastructure, and hard drives is so last week. So is there a way we can get rid of it all? Welcome serverless. Now we've covered serverless in a couple of other chapters in this course, but it is such an important part of modern cloud computing that we can do it again.

First of all, serverless is, of course, not serverless. Serverless refers to the fact that you aren't managing any servers yourself, at all. Your applications are running on someone else's server. Serverless is an extreme version of Platform as a Service, or PaaS. It means Azure will manage all the services you use. Let's have a look at three serverless products from Azure to get a sense of how it works and which solutions it is appropriate for.

1. Golden Oldie

   It is the first “serverless” service on Azure. The Azure service most will start with when exploring serverless architecture.

2. Single Task

   It is called function as only a single task is performed every time. The function only runs once for each invocation.

3. Basic Compute Unit

   It is a fundamental compute action and can be run millions of times per second if needed.

The first one is Azure Functions. This is the original serverless product on Azure and gives the clearest idea behind the technology. It's called a function because the entire idea is that the service provides a single function to do a single task. There are no servers, no applications, no systems. Just a single function. It is the most fundamental thing in compute, and Azure Functions will run your functions millions of times a minute or second if necessary.

Connect Systems

Connect systems both inside and outside of the Azure platform. Integrate not only apps, but also data flows, services and entire systems.

Automation

A lot of ways to schedule, automate and orchestrate tasks and processes.

Quick Start

No coding required to get started straight away. You just need access to your apps to integrate.

Logic Apps are a way to connect many different and varied systems both within and outside Azure. You can integrate apps, data, systems, and services across enterprises or organizations. A Logic App can schedule, automate, and orchestrate tasks, business processes, and workflows. For example, every time a tweet is made from an account, trigger some process. No coding skills are needed at all. All the connections and integrations are fully customizable in the Azure portal. So what does this mean in practice? For example, every time a new order is created in your ordering system, you can create a Logic App that creates a record for how long it took. It inserts that into a database and sends an email to the person in charge. You can also have a conditional path, so if, for example, the order was over $100, you also send a different email to customer service asking them to thank the customer. You can connect a ton of different applications either manually or using templates. There's no code needed, no servers to set up, and you can get started very quickly.

The last serverless product we'll go through is the Event Grid. But first, what is an event? In computing terms, it is when an application or process wants the world (being other connected applications) to know that something happened. It could be the placing of an order, for example. So when that order is placed, the ordering system raises an event. But how do you get that event to the applications that need to know? That is what Event Grid is for. Think of it as a big network of connections to any service that wants it. You can connect most Azure products to it, and when they're connected, events are sent to Event Grid from which connected services can receive the events. Event Grid is a routing service for connected applications, making sure events are sent and received fast and accurately. It's a serverless service because you don't manage any infrastructure. You merely sign up to use the service, which has a very specific and simple task. Well, simple in concept, at least. Event Grid makes setting up complex architecture of interconnected disparate services a lot simpler.

Event Grid is a routing service for connected applications, making sure events are sent and received fast and accurately. It's a serverless service because you don't manage any infrastructure. You merely sign up to use the service, which has a very specific and simple task. Well, simple in concept, at least. Event Grid makes setting up complex architecture of interconnected disparate services a lot simpler.

Event architecture is not part of the scope of this course, but it is a crucial part of cloud computing usage. Serverless solutions on Azure are a major reason to use the cloud platform. Removing the need to manage servers and infrastructure and simply focus on the immediate task of solving a business problem saves time, money, and gets you to market quicker. This is exactly where A Cloud Guru started. Our illustrious leader, Sam "Serverless" Kroonenburg, built the initial platform in just one month.

Routing Service

Event Grid is a routing service for sending and receiving events between applications.

Serverless

No management of infrastructure components. Create an Azure Event Grid and start connecting services.

Ease of Use

Event Grid Makes complex cloud architecture simpler.

Serverless approaches work extremely well with cloud services. ACG is all serverless.

• Yes, there are servers (they just aren’t yours).

• Azure Functions are the most basic of compute tasks. Does one function then dies until next time it is called.

• Logic Apps are a quick and simple way to integrate systems and applications inside and outside of Azure.

• Event Grid is a routing service for passing events between applications that need to know about the events.

DevOps is the work between development and production

u may have heard of the term DevOps before. It refers to the body of work between a developer finishing a feature or a section of code to the product being in production and real-world users are using it. So how does the software get into the real world, and how are processes for this created and managed? This is what DevOps is targeting.

The term DevOps is a mix of development and operations, and I'd love to give you an official definition, but there isn't one. Tough luck. Instead, I'll offer you my own description of the term. DevOps is, at its core about people - how developers, engineers, and system administrators organize themselves and work as a team to deliver better products faster. And in order to do this, Azure provides some tools and services that make this process easier.

The first tool is Azure DevOps, which is actually a platform consisting of five different tools. And these tools are meant to work together to provide a smooth, more robust, and ultimately faster development of new products. There's Azure Boards to allow project managers to keep track of work tasks, timelines, issues, and plan the full project timeline for a team. Azure Pipelines is for producing and testing your software and applications automatically every time a change is made. Azure Repos is a secure and managed way to store all your source code for your applications. This is the magic juice to developers. Azure Test Plans lets you set up automated tests of your applications to make sure it won't break once users start using it. Your quality assurance team will thank you for this part. And then Azure Artifacts, which is a way to share your applications and code libraries with other parties on your team or externally. And it's free. (Well, to get started with Azure DevOps at least.) And it is a platform that is part of an efficient and modern software product lifecycle.

Azure Boards

Keep track of work tasks, timelines, issues, planning and much more. Project managers love this part.

Azure Pipelines

Produce and test your software automatically and continuously.

Azure Repos

Store source code for your application securely in a managed way. Developers are big fans of this part.

Azure Test Plans

Design tests of applications to implement automatically. Quality assurance engineers love this bit.

Azure Artifacts

Share applications and code libraries with other teams inside and outside your organization.

The second tool we'll cover is Azure DevTest Labs. Where Azure DevOps focuses on the software and applications, DevTest Labs focuses on the environment they run on. Through centralized management, DevTest Labs can allow developers and engineers to create their own brand new test environments. This all happens without incurring unexpected costs, and, at the same time, minimizing waste of resources. You can even automate the creation of environments, which could allow tests of applications and development to be even more streamlined. Azure DevTest Labs comes with a range of features to let you tailor each environment to your needs and set up templates to reuse environments over and over again.

Environment Management

Allow developers and engineers to create environments for test and development.

Cost Management

You won’t incur unexpected costs and will even minimize waste of resources on your account.

Templates

Tailor your development and test environments and reuse them with templated deployments.

GitHub

• Acquired in 2018 by Microsoft.

• Code repository service for lots of big and small projects.

• A favourite among open-source communities.

• Microsoft themselves are one of the biggest users of GitHub.

GitHub Actions

• Very similar to Azure Pipelines.

• Build code, test code and publish code.

• Works with almost any platform, such as AWS, GCP and many more.

And this is where it gets a bit weird, if I'm honest. We've just covered Azure DevOps, but Microsoft also wants you to know about GitHub and GitHub Actions, which kind of do the same thing. GitHub was bought by Microsoft in 2018 and is, at its core, a code repository service. Lots of projects store their software code on GitHub to maintain it, keep it secure, share it, and manage it. GitHub is well known for hosting many open-source code repositories, meaning projects where you can see the code and often contribute to it. Microsoft is one of the biggest users of GitHub themselves, with all their major open-source projects, such as .NET Core and the Windows Terminal, hosted there. GitHub Actions is equivalent to Azure DevOps Pipelines. It allows you to build your code, test it, and then publish it to the live production environment when it passes all the tests. Just like Azure DevOps Pipelines, you can deploy to almost any kind of cloud platform and environment, such as AWS, Kubernetes, and Google Cloud Platform. Why does Microsoft have two products that do almost the same, I hear you ask? And that is a discussion for another time. Sorry. DevOps is an approach that is a fundamental part of modern software and technology development. On to the next lecture."

Internet of Things (IoT)

IoT is a network of millions of connected devices that function without human intervention. IoT Hub collates and manages data feeds as a PaaS product. IoT Central is a SaaS offering with templates and dashboards for a quick start.

Big Data

Processing and storage of very large data sets. Azure has Data Lake Analytics and HDInsight. This can lead to reduced cost, better decisions and new products.

Machine Learning

Use rules and models to train your AI implementation. Azure Machine Learning Studio has pre-made models and tools to help you get started. Azure Machine Learning Service is a collection of tools.

Serverless

You are using someone else’s servers. Azure Function is a single unit of compute. Azure Logic Apps can connect data feeds and applications. Azure Event Grid is a network to route events between applications.

DevOps

Azure DevOps has 5 tools: Boards, Pipelines, Repos, Test Plans and Artifacts. Use one or more in your workflow. Azure DevTest Labs lets you create full development and test environments easily and cost efficiently. GitHub and GitHub actions are tools that are similar to Azure DevOps.