Monday, 20 November 2017

Old but still very good, SlowCheetah with VSTS!

I was asked if there is a way of transforming configuration files at build time. Nothing else than the good, old and reliable SlowCheetah if you ask me Smile

Just install the Extension, and you will be able to add your transformations by right clicking the .config file and selecting Add Transform:

image

These are going to be based off the BuildConfiguration you defined in your Solution. Once this is done you can define your own settings, like this:

image

where a transformation can be something like this:

image

There are so many examples on how to do this, so please do not shoot on the pianist Smileall of that goes into a Version Control System of sorts, and it can be built by VSTS or TFS or any other Build Engine.

Your Build Definition needs to specify a Configuration, either at Queue time or embedded into the Build Definition itself.

imageimage

The result? As expected:

image

This isn’t really about VSTS or TFS per-se, but it is always a valuable approach to configuration management, and it was worth a refresher Smile

YAML Build Definitions in the Team Build, now what?

Among the news and announcements from Connect() you surely saw YAML Build Definitions mentioned, and you might wonder – what’s coming? How does this fit into the overall TFS/VSTS product?

Let’s start from the past, from 2011 – this UserVoice request asks for something that enables versioning for Build Definitions.

Being 2011 many things weren’t as available or as robust as of today, and the current Team Build was not remotely on the horizon. Fast-forward six years, and we’ve got YAML Build Definitions.

Didn’t we have a way of tracing Build Definitions without YAML? Really? Six years to implement some kind of traceability? Well…:

image

image

There is a built-in Compare Difference feature in both TFS and VSTS, and you can export your Build Definitions in a JSON file. So no, it is not just about traceability.

In the age of Continuous Delivery, Infrastructure as Code is critical. It saves an enormous amount of time and resources, and it is an extremely reliable way of automating the build and release process.

That is where YAML Build Definitions fit into the equation: they represent the concept of Infrastructure as Code pushed to the limit. You are not just treating the deployed infrastructure as code, you are also adding the deployment infrastructure in this definition, where as long as you have the required resources at your disposals (agent queues, build tasks, etc.) you are good to go.

This also does not mean the current Build Definitions are leaving – YAML is just for the Build Definitions, but the underlying technology is still the same. Also, with YAML you don’t get a visual breakdown of the tasks of the Build Definition:

imageimage

It is a different way of doing the same thing – it might not cater for everybody (I am a big fan of the current Definition UI because it makes it understandable for everybody regardless of the expertise and the role), but it adds options for someone who needs a different experience and has different requirements.

At the end of the day, the more the better Smile

Monday, 6 November 2017

Unorthodox reporting with TFS/VSTS and PowerQuery

I am a huge Excel fan, because it allows easy data transformation and its flexibility is second to none, despite its complexity. But I also use other tools, depending on the requirements.

Many users instead only use Excel, with no possible alternative. Good or bad, this is the norm in many organisations and trying to change this habit too early while trying to push new concepts or ideas only puts strain on these users, raising barriers and potentially preventing the very change we are pursuing.

That is where PowerQuery comes to the rescue. Ironically enough, I discovered it by accident trying to help my partner with some stuff from her work (she is a heavy Excel user) – PowerQuery is a very powerful data analysis engine (it goes hand to hand with PowerPivot, another Excel Data Modelling tool I am really fond of!) that, in a nutshell and from a developer point of view, enables database-like querying and reporting scenarios.

So what can you do with it? Well, let’s take a very easy example: you have a TFS/VSTS query which returns all the non-done PBIs in a backlog, and you want to report on this query so you will know how many Work Items you have in a certain state, but without using TFS or VSTS at all.

image

That is where Excel comes easily to the rescue: you can connect it to TFS/VSTS with the Team Add-in, downloading the raw data from the query you saved there:

image

image

Select the raw data you want to use and from the Data ribbon, select From Table. Excel will automatically recognise the data source you want to use, if you don’t select data beforehand you’d have to input the range manually.

image

PowerQuery now kicks in:

image

What we want to do is pretty easy and straightforward, so we are going to use Group By:

image

A basic group by works well here:

imageimage

If you use an advanced one you can group by based on multiple columns. If you have bugs as well as PBIs as requirements, that’s what you want:

image

image

Now, if you Close & Load, you are done. How is this useful in any way?

image

Easy: this query is going to show on the side of your spreadsheet:

image

Click on the Query you created, and you will be immediately shown the result:

imageor image

Moreover, it is quite dynamic.

Going deeper on it, the name isn’t cool at all – VSTS_<GUID> doesn’t say much. You can change it in the Query Editor:

image

Underneath you can see the Applied Steps – that is where things get interesting:

image

It is the visual representation of all the data transformation you applied. If you want to access these steps and change them, click on the Advanced Editor:

image

You will get the actual PowerQuery raw language (it is a functional language called M by the way Smile):

image

This is where you can start creating your custom transformations, leading to dynamic custom reports based on TFS/VSTS data.

image

image

I really find it cool and fascinating to be fair, bringing together such different user requirements and scenarios – without mentioning that you can up your percentage of Excel knowledge by a notch, which is always a great skill to master Smile

Friday, 13 October 2017

Help! I cannot complete my Pull Request in TFS!

A quick one today: if the Complete button is greyed out in your Pull Request UI, ask your TFS Administrator to start the TFS Job Agent.

image

The TFS Job Agent does many things – including handling Pull Requests completion.

Tuesday, 10 October 2017

Successfully handle disruptive changes with no downtime: the TFS 2015 example

It is something I’ve mentioned a few years ago but the question came out again during my presentation at x-celerate.de:

How should I handle a breaking change without service interruptions?

This is a brilliant question, and the best example I can give out is the TFS 2015 Upgrade that introduced support for renaming Team Projects.

If you don’t want to have downtime for your users the only mitigation is to introduce an intermediate migration layer which is going to be pouring data from the production stack and transform it into what you want.

The upside of this is that you are performing a very expensive and time-consuming operation out-of-band, so you can apply all the usual patterns for highly available application deployments.

The downside is that it is a costly operation, it could be compute, storage or something else but it will cost something out of it.

In my specific case I was able to perform a scheduled upgrade into the mandatory weekend window (yes, there was still a bit of downtime but it was due to the nature of the product and it was expected – you can overcome the hurdle if you are building your own product though) instead of having days of downtime due to the migration of data from a schema to another, at the cost of lots of storage space for the temporary tables and a dedicated server to run the tool.

Sunday, 8 October 2017

Re-release to an environment, don’t spin up a whole new deployment!

I know this happens on a regular basis – but it caught up my eye this morning as I am finishing up preparation for X-Celerate.de.

Let’s say your Release fails:

image

What many do is to actually spin up a whole new instance of the Release itself. While this works ok, you are missing out on something important: traceability.

In a sea of releases, with microservices and multiple moving pieces, how would you be able to trace back what happened during that failed release?

Why don’t you actually try to re-deploy the same failed bits instead?

image

Doing this provides you all the details about the previous failures, and it is going to be much easier to recall in case you might need to refer to the scenario in the future.

image

Oh in case you were wondering… it was all about my lab’s DNS server, which cached the Kudu website of an App Service I was deploying as 404 Smile but it is Sunday after all…

Thursday, 5 October 2017

If you don’t package your stuff you are doing it wrong!

Packages are a thing, exactly like containers are.

I mean – who wants to spend countless time in moving files, editing configuration files and the likes? Nobody, I know, but still so many people don’t take advantage of application packaging when it comes to deploying stuff!

Let’s take an average web application as an example. What is the reason that pushes you to actually move stuff from a certain folder (DLLs, .configs, etc.) to the target server, instead of packaging your application’s components and move these instead?

All you need to do is adding /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true to the MSBuild Arguments if you are using a traditional Build task.

image

If you are using .NET Core you just need to select the Zip Published Projects in the Publish task!

image

I just love the MSDeploy comeback. I am a huge supporter of this technology, because it makes life so much easier. It also has the side effect of enabling deployment to Azure in a snap, as it is one of the three supported delivery methods!

Let’s say you have a Build Definition and a Release Pipeline for this application. You want to deploy it to Azure – this is what you need to do:

imageWhat if you want to run the same application (providing it actually works on-premise as well and it does not specifically require any technology not available in your datacentre) on your own servers?

Firstly you’ll need to create a Deployment Group in VSTS – this can be done either by statically running the appropriate PowerShell script on your machines (interactively, via RDP) or dynamically with a bit of PowerShell or Azure if you are using IaaS. It is required because IaaS/on-premise machines in a Deployment Group will run an agent.

Then you can run whatever script you need to install the pre-requisites your application requires and configure all the settings, and eventually you can use the IIS Web App tasks to interact with IIS. Focusing on the IIS Web App Deploy task…

image

That Package is exactly the same package I used in the Azure deployment above. So you can easily have a Continuous Delivery pipeline on Azure and a different one (with the same cadence or a different one, your call) for on-premise, both starting from the same artifacts.

Containers are better – of course – but they require a minimum or ramp up or learning in order to actually implement them in a production environment. Moving to MSDeploy on the other hand is a matter of minutes at most, and it will provide a tangible improvement.

Friday, 29 September 2017

A few catches on customising the new Work Item form

If you use Team Foundation Server 2017 you already know this:

image

The new form is brilliant: it makes a much better use of the screen space, with a better UX in general.

But what if you have forms that were already using customised fields and a specific arrangement of controls?

image

The answer is that Microsoft uses a best-effort transformation system to automatically migrate your old form to the new, but for some reason I found myself in this situation – these two tabs won’t migrate to the new layout.

The new customisation model is brilliant – everything is now much cleaner and easier to use. All you need to do is add what you want to the <WebLayout> tag:

image

You can see that you now have Page, Group, Section as containers for controls, making life actually much easier when it comes to customisation. In my case I added two new Pages with the relevant control in there:

image

image

Of course you can customise the display layout by using the LayoutMode attribute. All the documentation is available here.