With the Service Fabric SDK 2.1.150 and Runtime 5.1.150 releases there is no longer a need for the build machine/agent to have Visual Studio and the Service Fabric SDK installed. There has also been a number of improvements to the tooling in Visual Studio Team Services for building and deploying Service Fabric applications.
I have created a video that runs through the creation of a basic Azure Service Fabric cluster using a standard ARM template, then going on to setup continuous integration and deployment with updates for a Service Fabric application.
Service Fabric Continuous Integration Documentation: https://azure.microsoft.com/en-gb/documentation/articles/service-fabric-set-up-continuous-integration/
Azure Quickstart Templates: https://github.com/Azure/azure-quickstart-templates/tree/master/service-fabric-secure-cluster-5-node-1-nodetype-wad
Service Fabric RP Helpers: https://github.com/ChackDan/Service-Fabric/tree/master/Scripts/ServiceFabricRPHelpers