Azure Services

FinOps for Azure: Optimizing Your Cloud Spend

In the cloud era, optimizing cloud spend has become a critical aspect of financial management. FinOps, a set of financial operations practices, empowers organizations to get the most out of their Azure investment. This blog post dives into the core principles of FinOps, explores the benefits it offers, and outlines practical strategies for implementing FinOps in your Azure environment.

Understanding the Cloud Cost Challenge

Traditional IT expenditure followed a capital expenditure (capex) model, where businesses purchased hardware and software upfront. Cloud computing introduces a paradigm shift with the operational expenditure (opex) model. Here, businesses pay for resources as they consume them, leading to variable and unpredictable costs.

FinOps tackles this challenge by providing a framework for managing cloud finances. It encompasses three key pillars:

• People: Establish a FinOps team or designate individuals responsible for overseeing cloud costs. This team should possess a blend of cloud technical expertise, financial acumen, and business process knowledge.
• Process: Define processes for budgeting, forecasting, and monitoring cloud expenses. This involves setting spending limits, creating chargeback models for different departments, and regularly reviewing cost reports.
• Tools: Leverage Azure Cost Management, a suite of tools that provides granular insights into your Azure spending. It enables cost allocation by resource, service, department, or any other relevant dimension.

It's essential to adopt a FinOps mindset that encourages collaboration between finance, IT, and business teams to drive cost efficiency and value realization in the cloud.

It's important to note that FinOps is not just about cost-cutting; it's about optimizing cloud spending to align with business objectives and maximize ROI.

Azure Cost Management: Optimizing Your Azure Spending

Azure Cost Management empowers you to analyze your Azure spending patterns and identify cost-saving opportunities. Here's a glimpse into its key functionalities:

• Cost Views: Generate comprehensive reports that categorize your Azure spending by various attributes like resource group, service, or department.
• Cost Alerts: Set up proactive alerts to notify you when your spending exceeds predefined thresholds.
• Reservations: Purchase reserved instances of frequently used Azure resources for significant upfront discounts.
• Recommendations: Azure Cost Management analyzes your usage patterns and recommends potential cost-saving measures, such as rightsizing resources or leveraging spot instances.

The Power of Tags and Azure Policy

Tags are metadata labels that you can attach to your Azure resources to categorize and track them effectively. They play a pivotal role in FinOps by enabling you to:

• Associate costs with specific departments, projects, or applications.
• Identify unused or underutilized resources for potential cost savings.
• Simplify cost allocation and chargeback processes.

Azure Policy helps enforce tagging standards and governance rules across your Azure environment. You can define policies that mandate specific tags for all resources, ensuring consistent cost allocation and data accuracy.

Benefits of Implementing FinOps

A well-defined FinOps strategy empowers you to:

• Gain Visibility into Cloud Spending: Obtain a clear picture of your Azure expenditures, enabling informed budgeting and cost control decisions.
• Optimize Cloud Costs: Identify and eliminate wasteful spending through cost-saving recommendations and proactive measures.
• Improve Cloud Governance: Enforce tagging policies and spending limits to ensure responsible cloud resource utilization.
• Align Cloud Spending with Business Value: Make data-driven decisions about cloud investments that support your business objectives.

Getting Started with FinOps in Azure

Implementing FinOps doesn't necessitate a complex overhaul. Here's a recommended approach:

1. Establish a FinOps Team: Assemble a cross-functional team comprising representatives from finance, IT, and business departments.
2. Set Clear Goals and Objectives: Define your FinOps goals, whether it's reducing costs by a specific percentage or improving budget forecasting accuracy.
3. Leverage Azure Cost Management: Start by exploring Azure Cost Management to understand your current spending patterns.
4. Implement Basic Tagging Standards: Enforce basic tagging policies to categorize your Azure resources for cost allocation purposes.
5. Continuously Monitor and Refine: Regularly review your cloud cost reports and identify areas for further optimization.

By following these steps and embracing a FinOps culture, you can effectively manage your Azure expenses and derive maximum value from your cloud investment.

Toolchain for FinOps in Azure

To streamline your FinOps practices in Azure, consider leveraging the following tools:

graph LR
A[Financial Operations Practices] --> B{Cloud Spend Optimization}
B --> C{Cost Visibility}
B --> D{Cost Optimization}
B --> E{Governance}
C --> F{Azure Cost Management}
D --> G{Azure Advisor}
E --> H{Azure Policy}
F --> I{Cost Views}
F --> J{Cost Alerts}
G --> K{Cost Recommendations}
H --> L{Tag Enforcement}

This toolchain combines Azure Cost Management, Azure Advisor, and Azure Policy to provide a comprehensive suite of capabilities for managing your Azure spending effectively.

Highly recommended, you can check FinOps toolkit, it's a set of tools and best practices to help you implement FinOps in your organization, it includes tools for cost allocation, budgeting, and forecasting, as well as best practices for FinOps implementation.

Conclusion

FinOps is an essential practice for organizations leveraging Azure. It empowers you to make informed decisions about your cloud finances, optimize spending, and achieve your business goals. As an Azure Solutions Architect, I recommend that you establish a FinOps practice within your organization to unlock the full potential of Azure and achieve financial efficiency in the cloud.

This blog post provides a foundational understanding of FinOps in Azure.

By embracing FinOps, you can transform your cloud cost management practices and drive business success in the cloud era.

References

• Thanks John Savill
• FinOps toolkit

Reduce your attack surface in Snowflake when using from Azure

When it comes to data security, reducing your attack surface is a crucial step. This post will guide you on how to minimize your attack surface when using Snowflake with Azure or Power BI.

What is Snowflake?

Snowflake is a cloud-based data warehousing platform that allows you to store and analyze large amounts of data. It is known for its scalability, performance, and ease of use. Snowflake is popular among organizations that need to process large volumes of data quickly and efficiently.

What is an Attack Surface?

An attack surface refers to the number of possible ways an attacker can get into a system and potentially extract data. The larger the attack surface, the more opportunities there are for attackers. Therefore, reducing the attack surface is a key aspect of securing your systems.

How to Reduce Your Attack Surface in Snowflake:

1. Use Azure Private Link Azure Private Link provides private connectivity from a virtual network to Snowflake, isolating your traffic from the public internet. It significantly reduces the attack surface by ensuring that traffic between Azure and Snowflake doesn't traverse over the public internet.

2. Implement Network Policies Snowflake allows you to define network policies that restrict access based on IP addresses. By limiting access to trusted IP ranges, you can reduce the potential points of entry for an attacker.

3. Enable Multi-Factor Authentication (MFA) MFA adds an extra layer of security by requiring users to provide at least two forms of identification before accessing the Snowflake account. This makes it harder for attackers to gain unauthorized access, even if they have your password.

In this blog post, we will show you how to reduce your attack surface when using Snowflake from Azure or Power BI by using Azure Private Link

Default Snowflake architecture

By default, Snowflake is accessible from the public internet. This means that anyone with the right credentials can access your Snowflake account from anywhere in the world, you can limit access to specific IP addresses, but this still exposes your Snowflake account to potential attackers.

This is a security risk because it exposes your Snowflake account to potential attackers.

Using Azure Private Link with Snowflake

Simple Architecture Diagram of Azure Private Link with Snowflake

The architecture of using Azure Private Link with Snowflake is as follows:

    graph TD
    A[Virtual Network] -->|Private Endpoint| B[Snowflake]
    B -->|Private Link Service| C[Private Link Resource]
    C -->|Private Connection| D[Virtual Network]
    D -->|Subnet| E[Private Endpoint]    

Architecture Components

Requirements

Before you can use Azure Private Link with Snowflake, you need to have the following requirements in place:

• A Snowflake account with ACCOUNTADMIN privileges
• Business Critical or higher Snowflake edition
• An Azure subscription with a Resource Group and privileges to create:
  • Virtual Network
  • Subnet
  • Private Endpoint

Step-by-Step Guide

Step 1: Retrieve Dedailts of your Snowflake Account

USE ROLE ACCOUNTADMIN;
select select SYSTEM$GET_PRIVATELINK_CONFIG();

Step 2: Create a Virtual Network

A virtual network is a private network that allows you to securely connect your resources in Azure. To create a virtual network with azcli, follow these steps:

az network vnet create \
  --name myVnet \
  --resource-group myResourceGroup \
  --address-prefix
  --subnet-name mySubnet \
  --subnet-prefix
  --enable-private-endpoint-network-policies true

Step 3: Create a Private Endpoint

The first step is to create a private endpoint in Azure. A private endpoint is a network interface that connects your virtual network to the Snowflake service. This allows you to access Snowflake using a private IP address, rather than a public one.

To create a private endpoint with azcli, follow these steps:

az network private-endpoint create \
  --name mySnowflakeEndpoint \
  --resource-group myResourceGroup \
  --vnet-name myVnet \
  --subnet mySubnet \
  --private-connection-resource-id /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Network/privateLinkServices/<Snowflake-service-name> \
  --connection-name mySnowflakeConnection

Check the status of the private endpoint:

az network private-endpoint show \
  --name mySnowflakeEndpoint \
  --resource-group myResourceGroup

Step 4: Authorize the Private Endpoint

The next step is to authorize the private endpoint to access the Snowflake service.

Retrieve the Resource ID of the private endpoint:

az network private-endpoint show \
  --name mySnowflakeEndpoint \
  --resource-group myResourceGroup

Create a temporary access token that Snowflake can use to authorize the private endpoint:

az account get-access-token --subscription <subscription-id>

Authorize the private endpoint in Snowflake:

USE ROLE ACCOUNTADMIN;
select SYSTEM$AUTHORIZE_PRIVATELINK('<resource-id>', '<access-token>');

Step 5: Block Public Access

To further reduce your attack surface, you can block public access to your Snowflake account. This ensures that all traffic to and from Snowflake goes through the private endpoint, reducing the risk of unauthorized access.

To block public access to your Snowflake account, you need to use Network Policy, follow these steps:

USE ROLE ACCOUNTADMIN;
CREATE NETWORK RULE allow_access_rule
  MODE = INGRESS
  TYPE = IPV4
  VALUE_LIST = ('192.168.1.99/24');

CREATE NETWORK RULE block_access_rule
  MODE = INGRESS
  TYPE = IPV4
  VALUE_LIST = ('0.0.0.0/0');

CREATE NETWORK POLICY public_network_policy
  ALLOWED_NETWORK_RULE_LIST = ('allow_access_rule')
  BLOCKED_NETWORK_RULE_LIST=('block_access_rule');

It's highly recommended to follow the best practices for network policies in Snowflake. You can find more information here: https://docs.snowflake.com/en/user-guide/network-policies#best-practices

Step 6: Test the Connection

To test the connection between your virtual network and Snowflake, you can use the SnowSQL client.

snowsql -a <account_name> -u <username> -r <role> -w <warehouse> -d <database> -s <schema>

Internal Stages with Azure Blob Private Endpoints

If you are using Azure Blob Storage as an internal stage in Snowflake, you can also use Azure Private Link to secure the connection between Snowflake and Azure Blob Storage.

It's recommended to use Azure Blob Storage with Private Endpoints to ensure that your data is secure and that you are reducing your attack surface, you can check the following documentation for more information: Azure Private Endpoints for Internal Stages to learn how to configure Azure Blob Storage with Private Endpoints in Snowflake.

Conclusion

Reducing your attack surface is a critical aspect of securing your systems. By using Azure Private Link with Snowflake, you can significantly reduce the risk of unauthorized access and data breaches. Follow the steps outlined in this blog post to set up Azure Private Link with Snowflake and start securing your data today.

References

• Azure Private Link documentation
• Snowflake Editions
• $GET_PRIVATELINK_CONFIG();
• Snowflake Network Policies
• Azure Private Endpoints for Internal Stages

Azure Container Registry: Artifact Cache

Azure Container Registry is a managed, private Docker registry service provided by Microsoft. It allows you to build, store, and manage container images and artifacts in a secure environment.

What is Artifact Caching?

Artifact Cache is a feature in Azure Container Registry that allows users to cache container images in a private container registry. It is available in Basic, Standard, and Premium service tiers.

Benefits of Artifact Cache

• Reliable pull operations: Faster pulls of container images are achievable by caching the container images in ACR.
• Private networks: Cached registries are available on private networks.
• Ensuring upstream content is delivered: Artifact Cache allows users to pull images from the local ACR instead of the upstream registry.

Limitations

Cache will only occur after at least one image pull is complete on the available container image

How to Use Artifact Cache in Azure Container Registry without credential?

Let's take a look at how you can implement artifact caching in Azure Container Registry.

Step 1: Create a Cache Rule

The first step is to create a cache rule in your Azure Container Registry. This rule specifies the source image that should be cached and the target image that will be stored in the cache.

az acr cache create -r MyRegistry -n MyRule -s docker.io/library/ubuntu -t ubuntu

Check the cache rule:

az acr cache show -r MyRegistry -n MyRule

Step 2: Pull the Image

Next, you need to pull the image from the source registry to the cache. This will download the image and store it in the cache for future use.

docker pull myregistry.azurecr.io/hello-world:latest

Step 3: Clean up the resources

Finally, you can clean up the resources by deleting the cache rule.

az acr cache delete -r MyRegistry -n MyRule

If you need to check other rules, you can use the following command:

az acr cache list -r MyRegistry

Conclusion

Azure Container Registry's Artifact Cache feature provides a convenient way to cache container images in a private registry, improving pull performance and reducing network traffic. By following the steps outlined in this article, you can easily set up and use artifact caching in your Azure Container Registry.

If you need to use the cache with authentication, you can use the following article: Enable Artifact Cache with authentication.

For more detailed information, please visit the official tutorial on the Microsoft Azure website.

References

• Azure Container Registry Documentation
• Azure Container Artifact Cache Overview

How to create a Management Group diagram with draw.io

I nedd to create a diagram of the Management Groups in Azure, and I remembered a project that did something similar but with PowerShell: https://github.com/PowerShellToday/new-mgmgroupdiagram.

Export your Management Group structure from Azure Portal or ask for it

If you can access the Azure Portal, you can export the Management Group structure to a CSV file. To do this, follow these steps:

1. Go to the Azure portal.
2. Navigate to Management groups.
3. Click on Export.
4. Save the CSV file to your local machine.

If you don't have access to the Azure Portal, you can ask your Azure administrator to export the Management Group structure for you.

The file has the following columns:

• id: The unique identifier of the Management Group or subscription.
• displayName: The name of the Management Group or subscription.
• itemType: The type of the item (Management Group or subscription).
• path: The path to the management or subscription group, its parent.
• accessLevel: Your access level.
• childSubscriptionCount: The number of child subscriptions at this level.
• totalSubscriptionCount: The total number of subscriptions.

Create a CSV to be imported into draw.io

1. Import the CSV file to excel, rename the sheet to "Export_Portal"
2. Create a second sheet with the following columns:
   • id: reference to the id in the first sheet
   • displayName: reference to the displayName in the first sheet
   • itemType: reference to the itemType in the first sheet
   • ParentId: Use the following formula to get the parent of the current item:

     =IF(ISERROR(FIND(","; Export_Portal!D2)); Export_Portal!D2; TRIM(RIGHT(SUBSTITUTE(Export_Portal!D2; ","; REPT(" "; LEN(Export_Portal!D2))); LEN(Export_Portal!D2))))
     

3. Export the second sheet to a CSV file.

Import the CSV file into draw.io

1. Go to draw.io and create a new diagram.
2. Click on Arrange > Insert > Advanced > CSV.

3. Insert the header for the columns: id, displayName, itemType, Parent:

       #label: %displayName%
       #stylename: itemType
       #styles: {"Management Group": "label;image=img/lib/azure2/general/Management_Groups.svg;whiteSpace=wrap;html=1;rounded=1; fillColor=%fill%;strokeColor=#6c8ebf;fillColor=#dae8fc;points=[[0.5,0,0,0,0],[0.5,1,0,0,0]];",\
       #"Subscription": "label;image=img/lib/azure2/general/Subscriptions.svg;whiteSpace=wrap;html=1;rounded=1; fillColor=%fill%;strokeColor=#d6b656;fillColor=#fff2cc;points=[[0.5,0,0,0,0],[0.5,1,0,0,0]];imageWidth=26;"}
       #
       #
       #namespace: csvimport-
       #
       #connect: {"from": "ParentId", "to": "displayName", "invert": true, "style": "curved=1;endArrow=blockThin;endFill=1;fontSize=11;edgeStyle=orthogonalEdgeStyle;"}
       #
       ## Node width and height, and padding for autosize
       #width: auto
       #height: auto
       #padding: -12
       #
       ## ignore: id,image,fill,stroke,refs,manager
       #
       ## Column to be renamed to link attribute (used as link).
       ## link: url
       #
       ## Spacing between nodes, heirarchical levels and parallel connections.
       #nodespacing: 40
       #levelspacing: 100
       #edgespacing: 40
       #
       ## layout: auto
       #layout: verticaltree
       #
       ## ---- CSV below this line. First line are column names. ----
   

   4. Paste the content of the CSV file and click on Import.

You should see a diagram with the Management Groups and Subscriptions.

For example:

This is the common structure for the Management Groups in the Enterprise Scale Landing Zone, now Accelerator Landing Zone:

    graph TD
        A[Root Management Group] --> B[Intermediary-Management-Group]
        B --> C[Decommissioned]
        B --> D[Landing Zones]
        B --> E[Platform]
        B --> F[Sandboxes]
        D --> G[Corp]
        D --> H[Online]
        E --> I[Connectivity]
        E --> J[Identity]
        E --> K[Management]        

And this is the CSV file to import into draw.io:

#label: %displayName%
#stylename: itemType
#styles: {"Management Group": "label;image=img/lib/azure2/general/Management_Groups.svg;whiteSpace=wrap;html=1;rounded=1; fillColor=%fill%;strokeColor=#6c8ebf;fillColor=#dae8fc;points=[[0.5,0,0,0,0],[0.5,1,0,0,0]];",\
#"Subscription": "label;image=img/lib/azure2/general/Subscriptions.svg;whiteSpace=wrap;html=1;rounded=1; fillColor=%fill%;strokeColor=#d6b656;fillColor=#fff2cc;points=[[0.5,0,0,0,0],[0.5,1,0,0,0]];imageWidth=26;"}
#
#
#namespace: csvimport-
#
#connect: {"from": "ParentId", "to": "displayName", "invert": true, "style": "curved=1;endArrow=blockThin;endFill=1;fontSize=11;edgeStyle=orthogonalEdgeStyle;"}
#
## Node width and height, and padding for autosize
#width: auto
#height: auto
#padding: -12
#
## ignore: id,image,fill,stroke,refs,manager
#
## Column to be renamed to link attribute (used as link).
## link: url
#
## Spacing between nodes, heirarchical levels and parallel connections.
#nodespacing: 40
#levelspacing: 100
#edgespacing: 40
#
## layout: auto
#layout: verticaltree
#
## ---- CSV below this line. First line are column names. ----
id,displayName,itemType,ParentId
1,Tenant Root Group,Management Group,
2,Intermediary Management Group,Management Group,Tenant Root Group
3,Decommissioned,Management Group,Intermediary Management Group
4,Landing Zones,Management Group,Intermediary Management Group
5,Platform,Management Group,Intermediary Management Group
6,Sandboxes,Management Group,Landing Zones
7,Corp,Management Group,Landing Zones
8,Online,Management Group,Landing Zones
9,Connectivity,Management Group,Platform
10,Identity,Management Group,Platform
11,Management,Management Group,Platform
12,subcr-1,Subscription,Decommissioned
13,subcr-2,Subscription,Sandboxes
14,subcr-3,Subscription,Corp
15,subcr-4,Subscription,Online
16,subcr-5,Subscription,Connectivity
17,subcr-6,Subscription,Identity
18,subcr-7,Subscription,Management

Make your diagram animated and interactive

You can make your diagram animated and interactive by following these steps:

1. File > Export as > URL
2. Add p=ex& after the first ? in the URL.

For example, the URL should look like this:

https://viewer.diagrams.net/?p=ex&tags=%7B%7D&highlight=0000ff&layers=1&nav=1&title=MGs.drawio#R7Zxbc5s4FMc%2FjR%2BbAQkEPK7dJHWn3XbW6exMX3ZkkLFakDxCvvXTr7jFxrZi1k0Wg5lxYnR0QfqfHxqOBB7AUbx5FHgx%2F8wDEg2AEWwG8P0AAGAgQ32llm1uMU0X5pZQ0KCw7QwT%2BosUxqJiuKQBSSoFJeeRpIuq0eeMEV9WbFgIvq4Wm%2FGoetYFDsmRYeLj6Nj6Nw3kvBwG8nYZHwgN58WpXeDkGTEuCxcjSeY44Os9E7wfwJHgXOZH8WZEolS9Upe83oMm97ljgjB5osK3hIgv0x%2BpJsCI8FQ5Jis0AHZAk0WEt3%2FimKhU3k7Z7HBLxq7x42mcwOHy%2B0dn9PCRvIOFz%2FYrZqWfCMMsPcFf6TCA8Sj4clE0KEn8tF0UBT9jpoSKCTso9TXr%2Fjgo%2B5YNKcI%2BmStHEbHvvKoMidyWLioGN6Rx7ksah0qFiE7Vf%2FxrKUgqSkgYEVjVfNh15Z%2BsI8ldsgpV9fVcdXmyUOdWbawVzso2l3Eqm6kOVVEWkKBIpSjRKBrxiIusF3aaTNWEw0QK%2FpPssiDyXTKdqZxqFRhg4s58ZV9wymQ2VnuoPsadcsrIqPzZqsLoOc88zFOftFOpfCsiJNloIdnX8pHwmEixVUWKCpYD7goituU1nCfXO%2FqB4ea2%2BR74dmHDhc%2FC57Z3TKqDwoFlco%2FS16fW0lA7ZpKImAQUZ0PXsXkJwS9cED3WDWINnQOsTegec%2B20g2tbw%2FV74vM4pklCOVMuvRjimtdHD3RzQJezcjlJ2%2BiIZtNrB81IQ%2FMnzALKFCLGd4Vz0uPcYZyhZ57nuSWzs6Ph%2BWuE5YyLuAe5wyB7rn0eZKsdILsakCdqYp7yze9MySfn9p7bBu8nAKxwCx3rmFvUDm49DbcjLn4jsuuRvTJkLWieRxa0AlnL0CD7hUWUkR7azkBre955aGE7oDW182y2FE9XVG4vR7d6u9wz2xyzjuN05d7AAhpmx4ESo%2Be1E7wexmBtnmN1G2%2F7TPbEtp1Y00Q1wq92rH9Zul23ZDn1xTvzFK%2BT5TTxBV1IytkpUk9tbLwxr%2FtdentWAzRFNjpmdTabAf8VWc0GXz7HANAr3dPWmG5bEofpdtZydsEl7B6slvXYXgu2wKrOuZYHWsutbg8t5xZewu1utaxH9lqQPVzyajOyum2yHFnrEmT3V8t6aK8FWmRYnYFWtyWWQ2tfAu3xalmP7rWg63hOZ9DV7Yrl6KJL0K0umvXYXgu2nmd0BVtbtzOWY%2Btcgu3h2lkP7rWAa5qu2yS5Ry9ZnGayGNcKR0tSog2qjPhLsXp2IGHBH%2BmrKCo5jbj%2F82lOWW5%2BSB2ZF5pxJovXX8ysUhCSSdEgF3LOQ85wdL%2BzFoKn5V6WW3WML4VPXhhS%2BYKHxCIk8qWC1mkHChJhdQ9T7cl%2FcEZN5UH3lLfqKm83qjy8YeVRo8pbN6y806jydveUR3WVdxtVHt2w8l6jyju3q3z%2BFF5jyrvdU96pq7wmLPiflPduWHnQpPLlrwPcpPKwUeU7GMPadZVvNIZFHYxh3brKNxrDog7GsF5d5RuNYVH3YtjylY3zyjcaw6LuxbDlewfnlW80hkXdi2HLp%2BfPK99oDIu6F8Naddfn7TeKYVVy90tVWd7eD37B%2B38B#%7B%22pageId%22%3A%22UGUHswWqf16rUITyRAQM%22%7D

You can check it here

References

• Automatically create draw.io diagrams from CSV files
• Animation and Automatic Layout: Explore Complex Diagrams

Moving Management Groups and Subscriptions

Managing your Azure resources efficiently often involves moving management groups and subscriptions. Here's a brief guide on how to do it:

Moving Management Groups

To move a management group, you need to have the necessary permissions. You must be an owner of the target parent management group and have Management Group Contributor role at the group you want to move.

Here's the step-by-step process:

1. Navigate to the Azure portal.
2. Go to Management groups.
3. Select the management group you want to move.
4. Click Details.
5. Under Parent group, click Change.
6. Choose the new parent group from the list and click Save.

Remember, moving a management group will also move all its child resources including other management groups and subscriptions.

Moving Subscriptions

You can move a subscription from one management group to another or within the same management group. To do this, you must have the Owner or Contributor role at the target management group and Owner role at the subscription level.

Follow these steps:

1. Go to the Azure portal.
2. Navigate to Management groups.
3. Select the management group where the subscription currently resides.
4. Click on Subscriptions.
5. Find the subscription you want to move and select ..." (More options).
6. Click Change parent.
7. In the pop-up window, select the new parent management group and click Save.

In conclusion, moving management groups and subscriptions allows for better organization and management of your Azure resources. However, it should be done carefully considering the impact on resources and compliance with assigned policies.

Management Groups

What are Management Groups?

Management Groups are a way to manage access, policies, and compliance for multiple subscriptions. They provide a way to manage access, policies, and compliance for multiple subscriptions. Management groups are containers that help you manage access, policy, and compliance for multiple subscriptions. You organize subscriptions into containers called "management groups" and apply your governance conditions to the management groups. All subscriptions within a management group automatically inherit the conditions applied to the management group.

Management Groups Hierarchy

The management group hierarchy is a level of management groups that represent the different levels of your organization. The hierarchy starts with a single root management group, which represents the Microsoft Entra ID tenant. The root management group is the highest level in the hierarchy. All other management groups are subgroups of the root management group.

Management group design considerations

When designing your management group hierarchy, consider the following:

• How does your organization differentiate services that are managed or run by particular teams?

• Are there any specific operations that need to be isolated due to business or regulatory compliance requirements?

• Management groups can be utilized to consolidate policy and initiative assignments through Azure Policy.

• A management group hierarchy can accommodate up to six nested levels. The tenant root level and the subscription level are not included in this count.

• Any principal, be it a user or service principal, within a Microsoft Entra tenant has the authority to establish new management groups. This is due to the fact that Azure role-based access control (RBAC) authorization for managing group activities is not activated by default. For additional details, refer to the guide on safeguarding your resource hierarchy.

• By default, all newly created subscriptions will be assigned to the tenant root management group.

Management group recommendations

• Maintain a relatively flat management group hierarchy, ideally with three to four levels maximum. This practice minimizes managerial complexity and overhead.

• Refrain from mirroring your organizational structure into a deeply nested management group hierarchy. Utilize management groups primarily for policy assignment rather than billing. This strategy aligns with the Azure landing zone conceptual architecture, which applies Azure policies to workloads that need similar security and compliance at the same management group level.

• Establish management groups under your root-level group representing different types of workloads you will host. These groups should reflect the security, compliance, connectivity, and feature requirements of the workloads. By doing this, you can apply a set of Azure policies at the management group level for all workloads with similar needs.

• Leverage resource tags for querying and horizontally traversing across the management group hierarchy. Resource tags, enforced or appended via Azure Policy, allow you to group resources for search purposes without relying on a complex management group hierarchy.

• Set up a top-level sandbox management group. This allows users to immediately experiment with Azure and try out resources not yet permitted in production environments. The sandbox provides isolation from your development, testing, and production settings.

• Create a platform management group beneath the root management group to support common platform policy and Azure role assignments. This ensures distinct policies can be applied to subscriptions used for your Azure foundation and centralizes billing for common resources in one foundational subscription set.

• Minimize the number of Azure Policy assignments made at the root management group scope. This reduces the debugging of inherited policies in lower-level management groups.

• Implement policies to enforce compliance requirements either at the management group or subscription scope to achieve policy-driven governance.

• Ensure only privileged users have operational access to management groups in the tenant. Enable Azure RBAC authorization in the management group hierarchy settings to fine-tune user privileges. By default, all users are authorized to create their own management groups under the root management group.

• Set up a default, dedicated management group for new subscriptions. This prevents any subscriptions from being placed under the root management group. This is particularly important if there are users eligible for Microsoft Developer Network (MSDN) or Visual Studio benefits and subscriptions. A sandbox management group could be a suitable candidate for this type of management group. For more information, see Setting - default management group.

• Avoid creating management groups for production, testing, and development environments. If needed, separate these groups into different subscriptions within the same management group.

Management Group Structure in the Enterprise Scale Landing Zone

This is the common structure for the Management Groups in the Enterprise Scale Landing Zone:

    graph TD
        A[Root Management Group] --> B[Intermediary-Management-Group]
        B --> C[Decommissioned]
        B --> D[Landing Zones]
        B --> E[Platform]
        B --> F[Sandboxes]
        D --> G[Corp]
        D --> H[Online]
        E --> I[Connectivity]
        E --> J[Identity]
        E --> K[Management]

1. Root Management Group
   • Intermediary-Management-Group
     • Decommissioned: This could be where resources that are being phased out or decommissioned are managed.
     • Sandboxes: This could be an area where developers can test and experiment without affecting production systems.
     • Landing Zones
       • Corp: This could represent corporate resources or applications.
       • Online: This could represent online or customer-facing applications.
     • Platform
       • Connectivity: This could manage resources related to network connectivity.
       • Identity: This could manage resources related to identity and access management.
       • Management: This could manage resources related to overall platform management.

This structure allows for clear segmentation of resources based on their purpose and lifecycle. For example, decommissioned resources are separated from active ones, like Sandbox, and resources within the 'Platform' are further categorized based on their function (Connectivity, Identity, Management). The 'Landing Zones' group appears to separate resources based on their use case or environment (Corp, Online).

The exact interpretation would depend on the specific context and conventions of your organization.

Bad Examples

Example 1: Deeply Nested Hierarchy

graph TD
    A[Root Management Group] --> B[Group 1]
    B --> C[Group 2]
    C --> D[Group 3]
    D --> E[Group 4]
    E --> F[Group 5]
    F --> G[Group 6]

Why it's bad: This hierarchy is too deep. It becomes difficult to manage and increases complexity. Azure supports up to six levels of nested management groups but it's recommended to keep the hierarchy as flat as possible for simplicity.

Example 2: Unorganized Structure

graph TD
    A[Root Management Group] --> B[Group 1]
    A --> C[Group 2]
    B --> D[Group 3]
    C --> E[Group 4]
    D --> F[Group 5]
    E --> G[Group 6]

Why it's bad: The structure is not well-organized and doesn't follow a logical grouping or hierarchy. This can lead to confusion and difficulty in managing resources and policies.

Example 3: Single Level Hierarchy

graph TD
    A[Root Management Group] --> B[Group 1]
    A --> C[Group 2]
    A --> D[Group 3]
    A --> E[Group 4]
    A --> F[Group 5]
    A --> G[Group 6]

Why it's bad: Although this structure is simple, it lacks the ability to group related subscriptions together under a common management group. This makes it harder to apply consistent policies across related subscriptions.

Example 4: Environment-Based Hierarchy

graph TD
    A[Root Management Group] --> B[Production Management Group]
    A[Root Management Group] --> C[Development Management Group]
    A[Root Management Group] --> D[Testing Management Group]

Why it's bad: This structure separates environments into different management groups, which can lead to duplication of policies and increased complexity. It's better to use subscriptions within the same management group to separate environments and apply policies accordingly.

Good examples

    graph TD
        A[Root Management Group] --> B[Intermediary-Management-Group]
        B --> C[Decommissioned]
        B --> D[Landing Zones]
        B --> E[Platform]
        B --> F[Sandboxes]
        D --> G[Corp]
        D --> H[Online]
        E --> I[Connectivity]
        E --> J[Identity]
        E --> K[Management]

References

• https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/landing-zone/design-area/resource-org-management-groups
• https://learn.microsoft.com/en-us/azure/governance/management-groups/overview

Azure Network, Hub-and-Spoke Topology

Hub and Spoke is a network topology where a central Hub is connected to multiple Spokes. The Hub acts as a central point of connectivity and control, while the Spokes are isolated networks that connect to the Hub. This topology is common in Azure to simplify the connectivity and management of virtual networks.

graph TD
    HUB(("Central Hub"))
    SPOKE1[Spoke1]
    SPOKE2[Spoke2]
    SPOKE3[Spoke3]
    SPOKEN[Spoke...]
    HUB --- SPOKE1
    HUB --- SPOKE2
    HUB --- SPOKE3
    HUB --- SPOKEN

Key Features of the Hub and Spoke Topology

1. Centralized Connectivity: The Hub centralizes the connectivity between the Spoke networks. This simplifies the administration and maintenance of the network.

2. Traffic Control: The Hub acts as a traffic control point between the Spoke networks. This allows for centralized application of security and routing policies.

3. Scalability: The Hub and Spoke topology is highly scalable and can grow to meet the organization's connectivity needs.

4. Resilience: The Hub and Spoke topology provides redundancy and resilience in case of network failures.

How to Use the Hub and Spoke Topology in Azure

To implement the Hub and Spoke topology in Azure, follow these steps:

# Step 1: Create a virtual network for the Hub
az network vnet create --name HubVnet --resource-group MyResourceGroup --location eastus --address-prefix

# Step 2: Create virtual networks for the Spokes
az network vnet create --name Spoke1Vnet --resource-group MyResourceGroup --location eastus --address-prefix
az network vnet create --name Spoke2Vnet --resource-group MyResourceGroup --location eastus --address-prefix
az network vnet create --name Spoke3Vnet --resource-group MyResourceGroup --location eastus --address-prefix

# Step 3: Connect the Spokes to the Hub
az network vnet peering create --name Spoke1ToHub --resource-group MyResourceGroup --vnet-name Spoke1Vnet --remote-vnet HubVnet --allow-vnet-access
az network vnet peering create --name Spoke2ToHub --resource-group MyResourceGroup --vnet-name Spoke2Vnet --remote-vnet HubVnet --allow-vnet-access
az network vnet peering create --name Spoke3ToHub --resource-group MyResourceGroup --vnet-name Spoke3Vnet --remote-vnet HubVnet --allow-vnet-access

# Step 4: Configure routing between the Hub and the Spokes
az network vnet peering update --name Spoke1ToHub --resource-group MyResourceGroup --vnet-name Spoke1Vnet --set virtualNetworkGateway:AllowGatewayTransit=true
az network vnet peering update --name Spoke2ToHub --resource-group MyResourceGroup --vnet-name Spoke2Vnet --set virtualNetworkGateway:AllowGatewayTransit=true
az network vnet peering update --name Spoke3ToHub --resource-group MyResourceGroup --vnet-name Spoke3Vnet --set virtualNetworkGateway:AllowGatewayTransit=true

# Step 5: Configure routing in the Hub
az network vnet peering update --name HubToSpoke1 --resource-group MyResourceGroup --vnet-name HubVnet --set virtualNetworkGateway:UseRemoteGateways=true
az network vnet peering update --name HubToSpoke2 --resource-group MyResourceGroup --vnet-name HubVnet --set virtualNetworkGateway:UseRemoteGateways=true
az network vnet peering update --name HubToSpoke3 --resource-group MyResourceGroup --vnet-name HubVnet --set virtualNetworkGateway:UseRemoteGateways=true

Variant of the Hub and Spoke Topology

A variant of the Hub and Spoke topology is the Hub and Spoke with peering between spokes that is generally used to allow direct connectivity between the Spoke networks without going through the Hub. This can be useful in scenarios where direct connectivity between the Spoke networks is required, such as data replication or application communication.

graph TD
    HUB(("Central Hub"))
    SPOKE1[Spoke1]
    SPOKE2[Spoke2]
    SPOKE3[Spoke3]
    SPOKEN[Spoke...]
    HUB --- SPOKE1
    HUB --- SPOKE2
    HUB --- SPOKE3
    HUB --- SPOKEN
    SPOKE1 -.- SPOKE2    

# Connect Spoke1 to Spoke2
az network vnet peering create --name Spoke1ToSpoke2 --resource-group MyResourceGroup --vnet-name Spoke1Vnet --remote-vnet Spoke2Vnet --allow-vnet-access

Scalability and Performance

The Hub and Spoke topology in Azure is highly scalable and can handle thousands of virtual networks and subnets. In terms of performance, the Hub and Spoke topology provides efficient and low-latency connectivity between the Spoke networks and the Hub.

Security and Compliance

The Hub and Spoke topology in Azure provides centralized control over network security and compliance. Security and routing policies can be applied centrally at the Hub, ensuring consistency and compliance with the organization's network policies.

Monitoring and Logging

Use Network Watcher to monitor and diagnose network problems in the Hub and Spoke topology. Network Watcher provides the following tools:

• Monitoring
  • Topology view shows you the resources in your virtual network and the relationships between them.
  • Connection monitor allows you to monitor connectivity and latency between endpoints within and outside of Azure.
• Network diagnostic tools
  • IP flow verify helps you detect traffic filtering issues at the virtual machine level.
  • NSG diagnostics helps you detect traffic filtering issues at the virtual machine, virtual machine scale set, or application gateway level.
  • Next hop helps you verify traffic routes and detect routing issues.
  • Connection troubleshoot enables a one-time check of connectivity and latency between a virtual machine and the Bastion host, application gateway, or another virtual machine.
  • Packet capture allows you to capture traffic from your virtual machine.
  • VPN troubleshoot runs multiple diagnostic checks on your gateways and VPN connections to help debug issues.
• Traffic
  • Network security group flow logs and virtual network flow logs let you log network traffic passing through your network security groups (NSGs) and virtual networks respectively.
  • Traffic analytics processes data from your network security group flow log allowing you to visualize, query, analyze, and understand your network traffic.

Virtual network flow logs have recently been released which allows for monitoring network traffic in Azure virtual networks.

Use Cases and Examples

The Hub and Spoke topology is ideal for organizations that require centralized connectivity and traffic control between multiple virtual networks in Azure. For example, an organization with multiple branches or departments can use the Hub and Spoke topology to securely and efficiently connect their virtual networks in the cloud.

Best Practices and Tips

When implementing the Hub and Spoke topology in Azure, it is recommended to follow these best practices:

• Security: Apply consistent security policies at the Hub and Spokes to ensure network protection.
• Resilience: Configure redundancy and resilience in the topology to ensure network availability in case of failures.
• Monitoring: Use monitoring tools like Azure Monitor to monitor network traffic and detect potential performance issues.

Conclusion

The Hub and Spoke topology is an effective way to simplify the connectivity and management of virtual networks in Azure. It provides centralized control over network connectivity and traffic, making it easier to implement security and routing policies consistently across the network. By following the recommended best practices and tips, organizations can make the most of the Hub and Spoke topology to meet their cloud connectivity needs.

References

• Network Watcher frequently asked questions (FAQ)
• Azure Topology

Kusto Query Language (KQL) for Azure Resource Graph

Azure Graph is a powerful tool provided by Microsoft to query data across all your Azure resources. It uses the Kusto Query Language (KQL), a read-only language similar to SQL, designed to query vast amounts of data in Azure services.

What is KQL?

KQL stands for Kusto Query Language. It's a request to process data and return results. The syntax is easy to read and author, making it ideal for data exploration and ad-hoc data mining tasks.

Using KQL with Azure Graph

Azure Graph allows you to use KQL to create complex queries that fetch information from your Azure resources. You can filter, sort, aggregate, and join data from different resources using KQL.

Here's an example of how you might use KQL to query Azure Graph:

Resources
| where type =~ 'microsoft.web/sites'
| project name, location, resourceGroup

This query retrieves all Azure Web Apps (websites) and projects their name, location, and resourceGroup.

Key Characteristics of KQL

1. Case Sensitivity: Unlike SQL, KQL is case-sensitive. So 'Name' and 'name' would be considered different identifiers.
2. Schema-Free: Kusto (Azure Data Explorer) doesn't require a fixed schema, allowing storage of diverse types of data.
3. Extensibility: While KQL has a wide array of functions, you can also create custom functions as per your needs.

Common Operators in KQL

• | : This operator creates a pipeline where the output of one command becomes the input of another.
• where : Filters rows based on specified conditions.
• summarize : Groups rows and calculates aggregate expressions.
• join : Combines rows from two tables based on a common column.
• project : Selects specific columns from the input.
• extend : Adds new columns to the input.
• order by : Sorts rows based on specified columns.

KQL Query Examples

1. List all Azure resources in a subscription

Resources

2. List all Azure resources in a resource group

Resources
| where resourceGroup == 'myResourceGroup'

3. List all Azure resources of a specific type

Resources
| where type =~ 'Microsoft.Compute/virtualMachines'

Pagination in KQL

KQL supports pagination using the limit and offset operators. You can use these operators to control the number of rows returned and skip a certain number of rows.

Resources
| limit 10
| offset 5

If you exceed payload limits, you can paginate Azure Resource Graph query results with powershell:

$kqlQuery = "policyResources | where type =~'Microsoft.Authorization/PolicySetDefinitions' or type =~'Microsoft.Authorization/PolicyDefinitions' | project definitionId = tolower(id), category = tostring(properties.metadata.category), definitionType = iff(type =~ 'Microsoft.Authorization/PolicysetDefinitions', 'initiative', 'policy'),PolicyDefinition=properties"

$batchSize = 5
$skipResult = 0

[System.Collections.Generic.List[string]]$kqlResult

while ($true) {

  if ($skipResult -gt 0) {
    $graphResult = Search-AzGraph -Query $kqlQuery -First $batchSize -SkipToken $graphResult.SkipToken
  }
  else {
    $graphResult = Search-AzGraph -Query $kqlQuery -First $batchSize
  }

  $kqlResult += $graphResult.data

  if ($graphResult.data.Count -lt $batchSize) {
    break;
  }
  $skipResult += $skipResult + $batchSize
}

Best Practices for Writing KQL Queries

1. Use project to Select Columns: Only select the columns you need to reduce the amount of data returned.
2. Use where to Filter Rows: Apply filters to reduce the number of rows processed.
3. Use summarize to Aggregate Data: Aggregate data to reduce the number of rows returned.
4. Use join to Combine Data: Combine data from different tables using the join operator.
5. Use order by to Sort Data: Sort data based on specific columns to make it easier to read.

Limitations of KQL

1. No DDL Operations: KQL doesn't support Data Definition Language (DDL) operations like creating tables or indexes.
2. No DML Operations: KQL doesn't support Data Manipulation Language (DML) operations like inserting, updating, or deleting data.
3. Limited Data Types: KQL has a limited set of data types compared to SQL.
4. No Transactions: KQL doesn't support transactions, so you can't group multiple operations into a single transaction.

Conclusion

KQL is a potent tool for querying large datasets in Azure. Its SQL-like syntax makes it accessible for anyone familiar with SQL, and its rich set of features makes it a flexible solution for a variety of data processing needs. Practice writing KQL queries to uncover valuable insights from your Azure resources!

References

• SQL to Kusto Query Language cheat sheet
• Demo for play

Cambio de nombres de los niveles de servicio de Microsoft Defender para Cloud

No es nuevo pero me gustaría recordar que Microsoft ha cambiado los nombres de los niveles de servicio de Microsoft Defender para Cloud. A continuación, se muestra una tabla con los nombres anteriores y los nuevos nombres de los niveles de servicio de Microsoft Defender para Cloud:

Azure Policy useful queries

Policy assignments and information about each of its respective definitions

// Policy assignments and information about each of its respective definitions
// Gets policy assignments in your environment with the respective assignment name,definition associated, category of definition (if applicable), as well as whether the definition type is an initiative or a single policy.

policyResources
| where type =~'Microsoft.Authorization/PolicyAssignments'
| project policyAssignmentId = tolower(tostring(id)), policyAssignmentDisplayName = tostring(properties.displayName), policyAssignmentDefinitionId = tolower(properties.policyDefinitionId)
| join kind=leftouter(
 policyResources
 | where type =~'Microsoft.Authorization/PolicySetDefinitions' or type =~'Microsoft.Authorization/PolicyDefinitions'
 | project definitionId = tolower(id), category = tostring(properties.metadata.category), definitionType = iff(type =~ 'Microsoft.Authorization/PolicysetDefinitions', 'initiative', 'policy')
) on $left.policyAssignmentDefinitionId == $right.definitionId

• Original Gist

List SubscriptionId and SubscriptionName

ResourceContainers
| where type =~ 'microsoft.resources/subscriptions'
| project subscriptionId, subscriptionName=name

List ManagementGroupId and ManagementGroupName

ResourceContainers
| where type =~ 'microsoft.management/managementgroups'
| project mgname = name, displayName = properties.displayName

Policy assignments and information about each of its respective definitions displaying the scope of the assignment, the subscription display name, the management group id, the resource group name, the definition type, the assignment name, the category of the definition, and the policy assignment ID.

policyResources
| where type =~'Microsoft.Authorization/PolicyAssignments'
| project policyAssignmentId = tolower(tostring(id)), policyAssignmentDisplayName = tostring(properties.displayName), policyAssignmentDefinitionId = tolower(properties.policyDefinitionId), subscriptionId = tostring(subscriptionId),resourceGroup=tostring(resourceGroup), AssignmentDefinition=properties
| join kind=leftouter(
    policyResources
    | where type =~'Microsoft.Authorization/PolicySetDefinitions' or type =~'Microsoft.Authorization/PolicyDefinitions'
    | project definitionId = tolower(id), category = tostring(properties.metadata.category), definitionType = iff(type =~ 'Microsoft.Authorization/PolicysetDefinitions', 'initiative', 'policy'),PolicyDefinition=properties
) on $left.policyAssignmentDefinitionId == $right.definitionId
| extend scope = iff(policyAssignmentId contains '/subscriptions/', 'Subscription', iff(policyAssignmentId contains '/providers/microsoft.management/managementgroups', 'Management Group', 'Resource Group'))
| join kind=leftouter (ResourceContainers
| where type =~ 'microsoft.resources/subscriptions'
| project subscriptionId, subscriptionName=name) on $left.subscriptionId == $right.subscriptionId
| extend SubscriptionDisplayName = iff(isnotempty(subscriptionId), subscriptionName, '')
| extend ManagementGroupName = iff(policyAssignmentId contains '/providers/microsoft.management/', split(policyAssignmentId, '/')[4],'')
| extend resourceGroupDisplayName = iff(isnotempty(resourceGroup), resourceGroup, '')
| project ManagementGroupName,SubscriptionDisplayName,resourceGroupDisplayName, scope,definitionType,policyAssignmentDisplayName, category,policyAssignmentId, AssignmentDefinition, PolicyDefinition

• Add Management Group Display Name