OMS Log Analytics – Azure Networking Analytics across different subscriptions

Update – 06-Feb-2017: I have worked with the Product Group to address the complexities and issues of configuring this across subscriptions. Azure Networking Analytics solution is now deprecated, and has been replaced with Azure Network Security Group Analytics solution.

This addresses the issue of having to write to a storage account, connecting the storage account to OMS, then configuring the solution in the Workspace to collect the data. This was a complex procedure, and in some cases the data would just not show up in Azure Networking Analytics. This update fixes these issues and now also uses the AzureDiagnostic query type.

Please make sure you are using the “Azure Network Security Group Analytics” solution moving forward, as pictured below.

Azure Network Security Group analytics

In this blog, we are going to use Azure Networking Analytics within Log Analytics in the Operations Management Suite to gain insights into Azure Network Security Groups across different subscriptions all within the same OMS Workspace. There are quite a lot of steps required to get this to work, and I found that some of the documented steps are spread across various different articles and some of the information did not work as it was described.

For example, the documentation states that there is not a requirement to push the data to a storage account, but in my testing I found that until it was pushed to a storage account and configured the Azure Networking Analytics solution within the OMS Workspace did not become active. I have documented all of the steps that I have verified enables you to monitor data across different subscriptions.

Azure Networking Analytics

This is currently a Preview solution within Log Analytics and provides a variety of insights into your Azure Network deployment, with new features likely to continue to be pushed out regularly. The following insights are currently available during Preview:

• Client and server errors reported by your application gateway
• Requests per hour per application gateway
• Failed requests per hour per application gateway
• Client and server errors by user agent
• Count of healthy and unhealthy hosts per application gateway
• Failed requests per application gateway
• Top network security rules that blocked the most number of flows within a specified time frame
• Top network security rules that allowed the most number of flows within a specified time frame
• Top MAC Addresses with the most number of blocked flows within a specified time frame
• Top MAC Addresses with the most number of allowed flows within a specified time frame

Adding the Solution to the OMS Workspace

After you have created your OMS Workspace you are able to add Solutions to your Workspace. Browse for Azure Networking Analytics and add it.

Enable collection of Azure Networking Analytics data on Network Security Groups

According to this link, there is not a requirement to write the Azure Networking Analytics to a storage account. However, unless I directed it to a storage account the Azure Networking Analytics never became active within the Portal and it was stuck with “Performing Assessment”. For me a combination of the following steps was required for this to work across subscriptions.

Store the Workspace Id

In the subscription where the OMS Workspace is configured, retrieve and store the WorkSpace resource object.

Enable Diagnostic on resource to be monitored

Change to the second subscription where the NSG is going to be configured.

After you have set it you can check the settings have been applied:

Possible Issues you may encounter at this stage

If the Microsoft.Insights Resource Provider is not registered in your subscription.

You just need to register the Resource Provider in your subscription

Secondly, to be able to set the Diagnostic setting for a resource to a OMS Workspace that resides in a different subscription, the subscriptions must be linked to the same Azure Activity Directory Tenant and the user must have administrative authority within both subscriptions. If this is not the case, you will receive the error below:

Check that the Intelligence Pack is enabled for the workspace

This should get done when you add the Azure Networking Analytics solution to the OMS Workspace, but it is good to check and confirm anyway. We are looking to confirm that AzureNetworking is Enabled (True)

Verify that data is being written to the storage account

At this stage, it is a good idea to confirm that diagnostic data is being written to our storage account before proceeding. You can verify this easily using Storage Explorer. Within the Storage Account you configured in the earlier steps we should see 2 containers:

  • insights-logs-networksecuritygroupeven
  • insights-logs-networksecuritygrouprulecounter

Drilling down viewing the JSON file shows the raw data, in this example we can see that the UserRule_SSH allow rule has 2 matched connections

Configure Log Analytics to read from the Storage Account

To be able to add Storage Accounts in different subscriptions we must use the full resource ID for the storage account.

Change to the subscription where you want to collect logs and store the details for the Storage Account

Create a new Storage Insight configuration

This is creating a Connected Data Source for an Azure Storage Account.

Verify that it has configured Azure Storage Account in the Connected Sources page.

Enable the Azure Networking Analytics in the Portal

Enable the Storage Insight

After the Storage Account is connected to the OMS Workspace, it appears as a final step is required to enable it for collection and processing. Until the last step is completed to enable the storage insight in the Log Analytics Workspace you will see the Azure Network Analytics will display the following message “Performing Assessment”. I left this at this stage for over 24 hours and it did not become active until the following cmdlets were run.

Run the following cmdlets to enable the storage insight:

There is also a User Interactive cmdlet you can run which will help step you through this step

Check the data in the OMS Workspace Portal
After all of the steps have been performed the data will start to appear in the portal, typically this takes 15-30 minutes.