Azure VM Network Bandwidth

July 2017 – Updated with new VPN Gateway types 

In this blog post we look at some network bandwidth tests for a variety of Azure VM sizes.

The tests have been run between two VM’s in the same VNet. Network bandwidth testing has been done with Linux using iperf3 running on CentOS 7.2 and Windows 2016 using the Ntttcp tool.

nettest1

Both single stream and multi stream tests were used. Of course your actual throughput numbers will vary from the ones seen in the tests below due to a number of factors (OS Type, workload characteristics etc….)

Note:  Microsoft is currently in the process of implementing some Azure Network optimisations:

    1. “Receive Side Scaling” – https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-optimize-network-bandwidth
    2. “Accelerated Networking” – https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-accelerated-networking-portal

The table below has been updated to include optimization 1. further updates will follow later in the year.
Entries in green include optimisation 1 “Receive Side Scaling”.

OS VM Type # Cores Single Stream Throughput ‘N’ Streams Throughput (1x #Cores)
CentOS (with 1) A0 Basic 0.25 10 Mbps 10 Mbps
CentOS (with 1) A1 Basic 1 100 Mbps 100 Mbps
CentOS (with 1) A2 Basic 2 200 Mbps 200 Mbps
CentOS (with 1) A3 Basic 4 400 Mbps 395 Mbps
CentOS (with 1)
A4 Basic 8 805 Mbps 816 Mbps
Windows A4 Basic 8 737 Mbps 734 Mbps
CentOS (with 1) A1 Standard 4 500 Mbps 500 Mbps
CentOS (with 1) A2 Standard 4 500 Mbps 500 Mbps
CentOS (with 1) A3 Standard 4 1000 Mbps 1000 Mbps
CentOS (with 1) A4 Standard 8 1990 Mbps 2000 Mbps
CentOS (with 1) A6 Standard 8 1000 Mbps 1000 Mbps
CentOS (with 1) A6 Standard 8 2000 Mbps 2000 Mbps
CentOS (with 1) A1 v2 1 495 Mbps 488 Mbps
Windows A1 v2 1 411 Mbps 467 Mbps
CentOS (with 1) A2 v2 2 500 Mbps 492 Mbps
CentOS (with 1) A4 v2 4 998 Mbps 999 Mbps
CentOS (with 1) A8 v2 8 1980 Mbps 1910 Mbps
CentOS (with 1) A8 8 4000 Mbps 4200 Mbps
CentOS (with 1) A9 16 4550 Mbps 7850 Mbps
CentOS (with 1) A10 8 3990 Mbps 3995 Mbps
Windows A10 8 1820 Mbps 3942 Mbps
CentOS (with 1) A11 16 4410 Mbps 8000 Mbps
CentOS (with 1) D1 v2 1 750 Mbps 726 Mbps
CentOS (with 1) D2 v2 2 1500 Mbps 1500 Mbps
CentOS (with 1) D3 v2 4 3000 Mbps 3000 Mbps
CentOS (with 1) D4 v2 8 4950 Mbps 6000 Mbps
CentOS (with 1) D5 v2 16 4840 Mbps 12000 Mbps
CentOS (with 1) D11 v2 2 1500 Mbps 1500 Mbps
CentOS (with 1) D12 v2 4 3000 Mbps 3000 Mbps
CentOS (with 1) D13 v2 8 4210 Mbps 5990 Mbps
CentOS (with 1) D14 v2 16 4990 Mbps 11900 Mbps
CentOS (with 1) D15 v2 20 4440 Mbps 15500 Mbps
Windows D15 v2 20 1002 Mbps 12176 Mbps
CentOS (with 1) F1 1 750 Mbps 749 Mbps
CentOS (with 1) F2 2 1500 Mbps 1490 Mbps
CentOS (with 1) F4 4 2990 Mbps 2995 Mbps
Windows (with 1) F4 4 928 Mbps 2640 Mbps
CentOS (with 1) F8 8 3490 Mbps 5990 Mbps
Windows F8 8 390 Mbps 4388 Mbps
CentOS (with 1) F16 16 4110 Mbps 11800 Mbps
Windows (with 1) F16 16 1096 Mbps 8416 Mbps
CentOS (with 1) G1 2 2000 Mbps 2000 Mbps
CentOS (with 1) G2 4 3270 Mbps 4000 Mbps
CentOS (with 1) G3 8 3160 Mbps 8000 Mbps
CentOS (with 1) G4 16 3970 Mbps 8880 Mbps
Windows G4 16 1602 Mbps 9488 Mbps
Windows (with 1) G4 16 1904 Mbps 7856 Mbps
CentOS (with 1) G5 32 3850 Mbps 13700 Mbps
CentOS (with 1) NV6 6 4850 Mbps 5970 Mbps
CentOS (with 1)
NV12 12 4760 Mbps 12200 Mbps

Test method

In each CentOS VM:

$ sudo yum -y update          (a very important step !)

Ensure Receive Side Scaling is enabled see:  https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-optimize-network-bandwidth

$ wget https://iperf.fr/download/fedora/iperf3-3.1.3-1.fc24.x86_64.rpm
$ sudo yum install iperf3-3.1.3-1.fc24.x86_64.rpm

On one VM run iperf in server mode
$ iperf3 -s

On another VM run single stream test:
$ iperf3 -c ip-of-server

For the multiple streams test:
$ iperf3 -c  ip-of-server  -P n

Where n =  number of cores in VM

In each Windows 2016 VM:

Apply the latest Windows updates then download ntttcp from here

On one VM (ip=w.x.y.z) run ntttcp in receiver mode
C:> ntttcp -r -m 1,*,w.x.y.z
&
C:> ntttcp -r -m n,*,w.x.y.z   (for the multi-thread test)
Where n = 8x number of cores in VM

On another VM run single thread test ntttcp in sender mode:
C:> ntttcp -s -m 1,*,w.x.y.z.

For the multi thread test:
C:> ntttcp -s -m n,*,w.x.y.z.

Where n =  the number of cores in VM

Peering VNets (Directly connected)

nettest2

Testing between VM’s in directly peered VNets showed no noticeable difference.

Peered VNets in the same region

Testing between VM’s indirectly peered via the new gateway types (VPMGW1, 2 & 3) shows bandwidth up to 2.8Gbs a big step forward from the previous gateway types that returned a maximum of  980 Mbps when using the now depreciated ‘High Performance’ gateway.

nettest3

Gateway Type Single Stream Throughput ‘N’ Stream Throughput (1x #Cores)
VPNGW1 700 Mbps 717 Mbps
VPNGW2 1400 Mbps 1430 Mbps
VPNGW3 1810 Mbps 2880 Mbps

For comparison here are the results form the now depreciated gateway types Standard and High Performance:

Gateway Type Single Stream Throughput ‘N’ Streams Throughput (1x #Cores)
Standard 472Mbps 580 Mbps
High Performance 720 Mbps 980 Mbps

Peered VNets via two BGP Gateways one in each region

nettest4

Gateway Type Single Stream Throughput ‘N’ Streams Throughput 
VPNGW1 550 Mbps 670 Mbps
VPNGW2 650 Mbps 780 Mbps
VPNGW3 650 Mbps 650 Mbps

For comparison here are the results form the now depreciated gateway types Standard and High Performance:

Gateway Type Single Stream Throughput ‘N’ Streams Throughput 
Standard 200 Mbps 280 Mbps
High Performance 210 Mbps 411 Mbps

Azure Application Proxy – Part 4

Setting up Azure Active Directory Sync

The first step is to add a “custom domain” to Azure Active Directory in preparation for directory synchronization.

In this sequence we will add our custom domain (in my case “andyt.work”) to our default directory we were assigned when we created a new Azure account.

Login to the “old” portal https://manage.windowsazure.com

Select Active Directory from the left hand icons and then ensure the Default Directory is selected.

appproxyp4-0-admin

Now ensure one of your cloud user has “Global Admin” rights over the directory, here I have a user called “admin” that has that capability:

appproxyp4-1-global

Now enter the “Domains” section of the Default Directory and Click the Add button at the bottom of the screen.

appproxyp4-2-add

Enter the domain name you own and click “add”appproxyp4-3-andyt

The next step involves making a TX record entry with domain registrar. The details of the TX record are shown on screen. There is also a link to a web page describing the process of adding the required TX record for various different domain registrars.

Once you have added the TX record and waited a while for it to propagate click “Verify”

appproxyp4-4-verify

Having completed the verify process you should see the domain as verified , you can now switch the assigned primary domain to your custom domain using the “Change Primary” button at the bottom of the screen.

appproxyp4-5-chpri

Specify your domain as the new primary and press the tick button to confirm

appproxyp4-6-chpri2

We now have registered our domain and made it the primary.

The next step is to synchronize our on premise active directory domain with the Azure hosted Active Directory.

Navigate to the Directory Integration section of the Default Directory and enable Directory Sync, and Save

appproxyp4-7-activate

Next download the AD connect tool to one of your domain controllers or a domain connected machine/VM from:

https://www.microsoft.com/en-us/download/details.aspx?id=47594

Run the Wizard

appproxyp4-8-exe

The AD Connect Wizard has an Express set up option which is sufficient for most cases

appproxyp4-9-wiz1

Enter the credentials of your Azure user that has Global Admin rights that we set up earlier (“admin” in my case).

appproxyp4-12-wiz4

Enter credentials of the local domain administrator

appproxyp4-13-wiz5

Finish by clicking Install

appproxyp4-14-wiz6

Back in the Azure portal you should now see any local users defined in your Active Directory domain also appearing in the users section of the Default Directory:

appproxyp4-16-synched

Your directory is now “synced” with Azure Active Directory, users can sign on to cloud based application using their username (email address) and the same password as they use on premise to sign onto your local domain.

Azure Application Proxy – Part 5

Enabling secure Access via Azure’s Application Proxy

Now we will make our Intranet web  application securely available over the public internet using Azure’s Application Proxy capability.
On the Windows server where the IIS server in running, start an elevated command prompt

appproxyp5-1-cmd

Note the hostname of the machine/VM that is running IIS and the domain.

appproxyp5-2-hostname

Run the command setspn to set up a service principal nam.

appproxyp5-3-setspn

Next login to the “old” Azure portal
https://manage.windowsazure.com

Select the Azure Active Directory icon on the left , select the Default Directory (or your active directory) and click the Add+ icon at the bottom.

appproxyp5-4-add

In the dialog box that pops up select the “Publish an application that will be accessible from outside your network” option.

appproxyp5-5-outside

Next enter a name for your application , the internal URL and the pre-authentication method you want. In this case we’re using Azure Active Directory as our local Active Directory is already sync’d with AD Connect to Azure Active Directory. (see Part 4).  (The other Pre Authentication option is Pass Through)

appproxyp5-6-addapp

Next we need to enable the Application Proxy

appproxyp5-7-enappproxy

appproxyp5-8-enabled

Then click the option to download the connector

appproxyp5-9-download

Once downloaded run the AADApplicationProxyConnectorInstaller

appproxyp5-10-exe

Follow the install process until the option to Run the connector troubleshooter appears, run the troubleshooter

appproxyp5-11-proxyinst

Conform all is ok – if not you may need to open some outbound ports

appproxyp5-12-troubleshoot

Check the Application Proxy Connector local services are running

appproxyp5-13-services

Back in the Azure portal

appproxyp5-14-running

Next in the portal navigate to your applications, select the Intranet application and use the configure option, make a note of the new external (Intranet) URL for your application.

appproxyp5-15-config1

Scroll down and set the SPN name we defined earlier using the setspn.exe command

appproxyp5-16-config2

If you’d like users to be able to see this application in their myapps.microsoft.com portal select the self-service option

appproxyp5-17-config3

Next – click the Users and Groups option, notice a default group has been created for this application, you can also use any of the other groups if you want to.

appproxyp5-18-group

Next navigate back to the Default Directory and select Groups and select the group automatically created for the Intranet application.

appproxyp5-19-usergroup1

Add any members you wish to this group

appproxyp5-20-usergroup2

Now try to access your application using the new external URL. Logging in using one of the users you allowed in the group above.

If this doesn’t work it may be because you have already logged in as a user who was not in the list of users your defined in the applications group

appproxyp5-21-noaccess

If required start a new in-private browser and login as one of the users defined in the group

appproxyp5-22-login

This should then log you in to your internal application from the external URL.

appproxyp5-23-loggedin

If you defined the self-service application then the application can also be listed on myapps.microsoft.com  (if it isn’t listed select Get more applications and select it form the list)

appproxyp5-24-myapp

And that’s it – we’re accessing an internal Intranet application using a public endpoint provided by Azure Application Proxy, our connection is authenticated using sync’d credentials and communication is securely setup between the on-premise application proxy connector and the Azure Cloud.

Azure Application Proxy – Part 3

Setting up the Application in IIS

Assuming you have just installed IIS. Stop the default web application.

appproxyp3-1-stop

Add a new website for our application.

appproxyp3-2-new

Specify a web site name – and select the path C:\www\intranet-app1 (the path we shared earlier in part 2).

appproxyp3-3-path

After clicking “OK” you will get a warning about port 80’s binding. This can be ignored as we have stopped the default web site that was using it.

appproxyp3-4-warning

At this point if you try to access the application it will show an authentication error:

appproxyp3-5-accesserror

We need to do some further setup to allow our chosen Authentication method. Enable kernel mode authentication on the Intranet-App1:

appproxyp3-6-auth1

Enable Windows Authentication

appproxyp3-7-auth2

Next Browse to the website – and you should be prompted to perform a domain login: (If you log straight in without being prompted, it may be because you are already logged into the domain, in that case start a private browsing session and retry.)

 

appproxyp3-8-browse1

You should now see the website page you created in Visual Studio

appproxyp3-9-browse2

Up to this point you have an application running on your Intranet, authenticating using your own Active Directory credentials.

In Part 4 we’ll enable Azure Application Proxy  to make this application securely available to external (Internet) users.

 

 

Azure Application Proxy – Part 2

To perform these steps, you’re going to need a domain joined Windows Server with IIS installed.

There are a few different ways to deploy a web application, perhaps the simplest is to deploy it using a file share.

Create a root folder for your IIS web site (C:\www) and within this root folder create and share a sub-folder for the intranet-app1 application we have just built:

appproxyp2-1-share

Map the share to ta network drive on the machine you’ve used for Visual Studio

appproxyp2-1a-share2

In the main Visual Studio window – select the Intranet-app1, right click and select Publish …

appproxyp2-2-publish1

In the Publish dialog box select the Custom option, enter a new custom profile name, click OK and then click Next

appproxyp2-3-publish2

Now select File System as the publish method and set the target location to the file share network drive location we set up previously. Click Publish

appproxyp2-4-publish3

If you now resize the Output window in Visual Studio you should see the web application deploying successfully.

appproxyp2-5-publish4

In Part 3 we’ll configure the IIS server to support our authentication choice.

 

Azure Application Proxy – In Action

This blog post demonstrates how to use Azure Application Proxy.

Azure Application Proxy enables you to take an internal web application and make it securely available outside of your organisation. A few different authentication options can be enabled for your internal application to help secure it:

  • If your application does not use and form of sign-in then Azures Active Directory (AAD) sign-in can be added to the public endpoint Azure Application Proxy provides.
  • Pass through, relying on you on premise  authentication.
  • If your application does use Active Directory sign-in then you have the option to set up and use AAD based single sign-on. This post demonstrates that option.

If you would like to test the scenario described above, you may want to firstly create a simple application rather than using a real application. Part 1 of this blog shows how to do that using Visual Studio. If you already have a web application that authenticated against your local Active Directory, you can skip Parts 1,2 and 3. If you already have your Azure Active Directory synchronized with your local domain you can skip through Part 4 as well.

Part 1 – Creating a simple application with Visual Studio

Part 2 – Publishing the Application to a local IIS server

Part 3 – Setting up IIS for Authentication

Part 4 – Set up your Local Domain and Directory Synchronization

Part 5 – Enable Azure Application Proxy

 

 

 

 

 

Azure Application Proxy – Part 1

Part 1. Creating a simple application in Visual Studio

If you don’t already have Visual Studio you can download and install the community edition for free from here.

Start Visual Studio and create a simple web application:

When Visual Studio has started select  File -> New -> Project 

appproxy0-vizstudio

Then select a ASP.NET Web Application, ensure Application Insights is not selected, enter a name for your project (“intranet-app1” in my case) and click OK.

appproxy1-vizstudio

On the next screen select Web Forms unselect the Host in the Cloud button if it is selected and then click the Change Authentication button.

appproxy2-vizstudio

In the Change Authentication dialog – select the Windows Authentication option and click OK

appproxy3-vizstudio

Back on the select a template box click OK

appproxy4-vizstudio

Next on the main Visual Studio workspace select Default.aspx from the Solution Explorer window on the right hand side. Right click and select View Designer

appproxy5-vizstudio

In the main window for Default.aspx – click around ASP.NET and change the default text to something you will recognize.

appproxy6-vizstudio

Finally Build (or Rebuild) the solution

appproxy7-vizstudio

Now we’re ready to Publish this application which is covered in Part 2