A client machine was installed with Azure CLI and was configured to operate on the Azure cloud Kubernetes cluster with kubectl (Reference 10). The VM instance type chosen was “Standard_D2s_v5” which includes vCPUs based on 2 x Intel(R) Xeon(R) Platinum 8370C (3rd generation Xeon scalable processors, codename Ice Lake). A single Azure VM (node pool with 1 node) was used for this deployment. The output of the inference script is shown below in figure 7. A test client container pod was used to launch the demo inference script.
Onboard the Azure VM as an Azure Arc-enabled server
At present there are only a couple of Azure services that are generally available (GA) with many still in preview. For instance, once a virtual machine (VM) is projected in to Azure it looks and feels very much like an Azure VM, you can then view it along side all of your other Azure VMs. It is important to understand that Azure Arc is https://chickenroadapp.in/ not single service, it is not something you deploy and boom you have all of the capabilities of Azure Arc and are charged for a single service.
To get started, we’ll first provision a new Azure VM in a dedicated resource group within the sandboxed subscription. However, there are a few steps you can take as a workaround to onboard an Azure VM running Windows Server as an Arc-enabled server. ⚠️ Keep in mind that this blog post and the steps to configure an Azure VM as an Arc-enabled server are meant exclusively for testing and research. You can go ahead and Azure Arc enable servers or use Azure Arc enabled VMware vSphere and get centralized visibility of your assets, tag them, place them in resource groups etc at no extra cost. The Data Science team will then be able to discover a list of available compute targets and instance types in the Machine learning workspace and these can be used for training or inference workloads.
- You can also configure clusters and deploy applications using GitOps based config management.
- There are services which are free for Azure VMs such as Azure Policy Guest Configuration and Azure Automation services like Update Management which are charged for per server for Azure Arc enabled Servers.
- While it’s strongly not advised to install Azure Arc-enabled servers on an Azure VM for production scenarios, it is possible to configure Azure Arc-enabled servers on an Azure VM for training and testing purposes only.
- The key benefit is seamlessly deploying Intel OpenVINO software that is highly optimized for Intel Xeon processors at their on premise or cloud or edge locations in a consistent manner.
- It enables organisations to take advantage of cloud practices, services and automation in their datacentre or at the edge.
Azure Arc Pricing
To use the script, start by saving a copy as “Create-Azure-Windows-VM-with-VNet-and-RG-for-Azure-Arc-learning.ps1” or downloading it directly from GitHub. You can either use the Azure Portal or, for a faster setup, run the Azure PowerShell script below. In this blog post, I’ll walk you through the easy setup process using Azure PowerShell, a PowerShell script, and the free Azure Bastion Developer SKU. This can be particularly useful for testing or learning purposes when you don’t have access to any on-premises machines. In any normal scenario, you can’t connect an Azure VM as an Azure Arc-enabled server because it’s already represented in Azure Resource Manager, with all native Azure features and capabilities easily accessible if needed. These professionals have the options to create their own models or use models built from open-source platforms like PyTorch.
Applications or services packaged as containers can be deployed and managed with the same Kubernetes based eco-system tools in the public cloud, on premise or Edge locations. In this article we demonstrated Intel OpenVINO based inference deployment on 3rd Generation Intel Xeon processors, both on-premise and in Azure cloud with Kubernetes. The consistency and mobility of deploying inference applications using OpenVINO in a hybrid Kubernetes based cloud environment was demonstrated.
Azure GitOps with Flux enables deploying applications on Arc enabled Kubernetes clusters with Helm Charts (Reference 11). The Azure Kubernetes instance details are shown in figure below. This is shown in Azure Portal figure below (Resource Group is “ArcK8s” and Kubernetes instances are “AzureK8s” and “minikube”)
Azure Arc-enabled Application Services
This then allows you to perform some VM operations from the Azure portal like start/stop/restart, as well as self-service access which you can control using Azure Role Based Access Control. When you’re connected an automatic discovery of inventory of vSphere resources takes place. This is achieved by deploying the Azure Arc Resource Bridge in a vSphere environment.
The same steps used for the on premise minikube instance was used for OVMS deployment. A Kubernetes cluster was deployed in Azure cloud using Azure Kubernetes service. The local minikube based Kubernetes deployment was Azure Arc enabled via a connection procedure to the Azure cloud (Reference 6). OVMS is based on the same architecture as TensorFlow Serving and Inference as a service is provided via gRPC or REST API, making it easy to consume by applications that require AI model inference.
AZURE CLOUD DEPLOYMENT
Indirectly connected then the view from the Azure portal is read only, this allows you to see an inventory of your SQL managed instances but you can’t do anything with them from the portal and local tools must be used to make changes. Directly connected of course offers the most benefits and allows users to use Azure Resource manager (ARM) APIs, Azure CLI and the Azure portal and is a similar experience to using the service in Azure. Currently within this category the only service which is GA is Azure SQL Managed Instance. These Kubernetes clusters appear as Azure Resources, becoming a platform to deploy Azure Services to. Azure Arc enabled Kubernetes works with any Cloud Native Computing Foundation (CNCF) certified Kubernetes clusters and this includes clusters running in other cloud native platforms such as AWS and GCP.
For simplicity, we used local host VM storage to host the model data and attach to the deployed OVMS pod via Kubernetes “hostPath” mechanism. On the Azure portal, the minkube Kubernetes deployment (single node cluster) can be managed under the created resource group “ArcK8s” as shown in screenshot figures below. For on premises production Kubernetes deployment, this could be Azure Kubernetes Service deployed on Azure Stack HCI, VMware Tanzu on VMware vSphere or RedHat OpenShift or other supported Kubernetes platforms. A virtual machine was setup and installed with a single node Kubernetes deployment using minikube (Reference 5). The highlighted capability provides Enterprises the choice to deploy optimized Intel AI Inferencing applications either on cloud or on premise or both based on their business needs and available capacity.
- Rather it is a set of technologies and capabilities from which you can pick and choose features to reach your desired outcome based on your requirements, you might say it’s adaptive.
- To manage your Azure VM as an Azure Arc-enabled server, you first need to make a few changes to the VM before installing and configuring it.
- This category is about bringing Azure Services to wherever your workloads need to run, ensuring that these environments can also take advantage of cloud native innovation.
- The VM instance type chosen was “Standard_D2s_v5” which includes vCPUs based on 2 x Intel(R) Xeon(R) Platinum 8370C (3rd generation Xeon scalable processors, codename Ice Lake).
This application development and deployment model enables scale and agility in today’s hybrid and multi-cloud environments. This script will create a virtual network (VNet) in a specified resource group, deploy an Azure VM running Windows Server 2022 within the same resource group, and connect the VM to a subnet in the VNet. Keep in might you need an Arc-enabled Kubernetes platform to deploy these services too. There are services which are free for Azure VMs such as Azure Policy Guest Configuration and Azure Automation services like Update Management which are charged for per server for Azure Arc enabled Servers. If you want to take advantage of services such as Azure Monitor etc on your Arc connected servers then these will be charged just as if you were using these on your Azure VMs are they are an Azure Service. Azure Machine learning can be used by Machine Learning (ML) professionals, data scientists and engineers in their day to day workflows to train and deploy models as well as manage machine learning operations.
To manage your Azure VM as an Azure Arc-enabled server, you first need to make a few changes to the VM before installing and configuring it. Then, click Create to deploy your Bastion Developer host. To set it up, log in to the Azure Portal, and in the global search bar, type in “bastion”. Currently, deploying this Bastion SKU with Azure PowerShell is not possible. I prefer using Azure Bastion for this, and fortunately, for Dev/Test purposes, you can now easily set up an Azure Bastion Developer, which is a free, lightweight version of the Azure Bastion service.
If you followed the previous steps to deploy a new VM, no extensions, such as the Azure Monitor agent, should be installed. First, all VM extensions deployed to the Azure VM should be removed. Select the correct subscription and resource group, provide a name, and choose the appropriate region.
Azure Arc-enabled Infrastructure
As a result, Azure management services can now be extended to this non-Azure machine. Once connected, the server becomes what is known as a connected machine and treated as a resource in Azure. It enables organisations to take advantage of cloud practices, services and automation in their datacentre or at the edge.
Hybrid AI Inferencing managed with Microsoft Azure Arc-Enabled Kubernetes
This category is about gaining central visibility of all of your servers wherever they live and then as appropriate applying consistent Azure governance, management and security practices across all of those servers. You can now purchase ESUs on a monthly subscription basis via Azure Arc. This avoids having to pay for ESUs on an annual basis. It also auto discovers all EC2 instances in AWS and onboards them to Azure Arc by installing the Azure Connected Machine Agent. You can also monitor these servers with Azure Monitor and use Azure Update Manager in Azure Automation to manage operating system updates for your server just like you do in Azure. These connected machines can be enhanced from a security perspective by using Azure Defender or Microsoft Sentinal.
Azure Arc-enabled Servers
The inference request included specifying the Resnet50 model name to be used for inference. The image classification script was used that submits a list of images one at a time to OVMS for inference. The OVMS deployment can verified with a demo Inference application. The Kubernetes OVMS deployed state details are shown in figure below. As a proof-of-concept, a Kubernetes deployment was setup at one of the Intel onsite lab locations. This benefit comes along with utilizing a “single pane” for management and application deployment of disperse Kubernetes clusters from Azure Cloud as show in Solution Overview Figure 1 below.
If you have multiple Azure tenants, use the Set-AzContext -tenantID cmdlet to select the correct tenant before executing the script. Before running the script, make sure to sign in with the Connect-AzAccount cmdlet to link your Azure account. Adjust the dynamic and other variables to fit your specific needs, and then run the script using Windows Terminal, Visual Studio Code, or Windows PowerShell.
Alongside using Azure management constructs like Tags and Resource Groups, Azure Policies can also be used to govern the server as you do your Azure Servers. It is assigned a Resource ID which enables it to be tagged and placed in a resource group. This category is about bringing Azure Services to wherever your workloads need to run, ensuring that these environments can also take advantage of cloud native innovation. Rather it is a set of technologies and capabilities from which you can pick and choose features to reach your desired outcome based on your requirements, you might say it’s adaptive. Organisations can make use of role based access control to manage and delegate access to teams across the business. This may look like applications running across different infrastructure and hosting environments as well as multiple public cloud environments.
The host node /models folder will be mapped to /models folder within the deployed container pod. This method was not used for this deployment since minkube Kubernetes version was incompatible with resource types required for Azure GitOps agents. Further both deployments uses the same tools, install methods and were monitored centrally from Azure cloud. Instead of local VM node, steps were performed on the Azure cloud VM node and the Azure cloud Kubernetes instance. The Azure Kubernetes instance was deployed under the same Azure Resource Group as the earlier minikube local Kubernetes instance. The steps for launching the test client pod and executing the demo inference script is provided in the Appendix.
The key benefit is seamlessly deploying Intel OpenVINO software that is highly optimized for Intel Xeon processors at their on premise or cloud or edge locations in a consistent manner. OpenVINO toolkit includes pre-trained and optimized models to enable inference applications. Intel OpenVINO™ is an open-source software toolkit for optimizing and deploying AI inference across a variety of Intel CPU and accelerator devices (Reference 3). Cloud native deployment with Kubernetes orchestration has enabled the “Write Once, Deploy Anywhere” paradigm for applications. If you have any questions or suggestions regarding this blog post and the use scripts, feel free to reach out to me via my X handle (@wmatthyssen) or leave a comment below. In this blog post, I’ll show you how to accomplish this using Azure PowerShell, a standard PowerShell script, and the Azure Bastion Developer host to configure everything in an easy and free way.
