Using TeamCity for continuous integration with NodeJS and Grunt

CogUpdate 2023: Over time, we ended our journey with TeamCity and moved to the cloud-based Azure DevOps closer to our development stack. If this is closer to your stack (or you want to know more), check out our Azure tutorials and other blog posts:

- How to Setup a Private NuGet Feed in Azure DevOps Pipelines
- Generate Free SSL Certificates and Bind It to Azure Webapp With DevOps
- All Cogworks blog posts 

What is TeamCity and Continuous Integration?

TeamCity is a tool developed by JetBrains to run a continuous integration environment. TeamCity allows the development team to push to a Git repository and for the TeamCity tool to monitor the Git repository and perform a series of build steps for the website to be made and deployed to a web server. In our case, at Cogworks, a website in our development environment. 

This allows the code to be checked, ensuring it compiles and builds as it would inside Visual Studio. You can then be sure that a working site is deployed.

With TeamCity, you can be as simple or fancy as you like, involving building steps that copy files from one place to another or running complex tasks such as utilising tools to help you migrate databases from one environment to another—plenty of scenarios and ideas that can be set up to fit your team's development workflow.

What are NodeJS, Compass, Grunt and Bower?

Before I go on to explain this, I'd like to add a disclaimer that I claim to be no expert in this field and someone with more experience with these tools may give a better explanation of their use. So, as I am still new to these front-end toolsets, I'll try my best to explain them.

So, let's start with NodeJS. This Javascript application framework allows you to build a whole range of applications ranging from a HTTP server written in Javascript to slightly more geeky things such as controlling Parrot's AR Drone. Some clever bodies have also written Grunt and Bower in NodeJS, hence the need for it in this scenario. So, from what I can tell, your imagination is the only limit with NodeJS, and it's worth browsing the NPM repository to see what other people are up to.

Next, onto Compass, this is seen as a standard and valuable extension to the popular CSS preprocessor Sass. It also gives some extra functionality to your Sass files, such as the power to automatically create a sprite image and use it in your CSS - It's voodoo magic!!

Grunt is what all the front-end devs are getting excited about now. This is because Grunt is a task runner for their front-end workflows. You can compare this to your C# MSBuild workflows. So front-end devs can run tasks such as 'Watch', automatically monitoring changes to Sass and CoffeeScript source files. These are then automatically re-compiled into CSS and JavaScript files, and the browser reloads. Other common workflows are to use it as a build runner and for CSS and JS files to be minified and concatenated together, along with tasks such as running all images through an image minifier to ensure they are optimised and compressed for the best browsing experience. Again, there are a whole heap of uses for front-end devs, and it will be exciting to see what the future holds for Grunt.

Finally, Bower is a front-end package manager who is by the Twitter guys and runs on NodeJS. This fetches other front-end tools you may wish to use. For example, you may use Bower to install and bring the latest version of jQuery, jquery-ui and jQuery.cookies, along with a range of other client-side dependencies. This is similar to the NodeJS NPM repository but is aimed at web dependencies from jQuery to handlebars.

Why do we need all these frameworks?

All these tools allow us to write better front-end code. It allows us to compress images and minify large JS and CSS files. It helps automatically improve the overall quality of the browsing experience on as many devices and connection speeds as possible.

They help us reduce some of the hard work by allowing us to automate a lot of the front-end build processes.

You can think of Grunt as the front-end developer equivalent of a build process as we do as .NET Developers.

How to install what you need to get apps working with TeamCity.

So, I recently went through this process and made detailed notes about how I installed and set up what was needed to get all these apps working nicely with TeamCity.

So here goes:

Git

First, install Git and go through the installer. The most important steps are as follows:

From the screenshot above, you must ensure you chose to run Git from the Windows command prompt. This ensures you can type Git in any console window as it has been added to your computer's PATH. Without doing this, you will make your life hard if you work with things like bower.io, which expects Git to be in the PATH.

The next thing to ensure your life is easier in the future if you choose the Windows line ending from this step in the installer:

Now that we have Git installed, let's quickly check and ensure it is installed correctly.

From a console window, type the following git -v and hit enter. You should see a version number displayed if git has been successfully installed.

Installing NodeJS

The next part I needed to install on the server was NodeJS, which allows a whole heap of extensions to be installed via NPM (Node Package Manager). Think of this as the equivalent of NuGet in the .NET world, specifically for NodeJS.

Again, as with the Git install, there is a crucial step to ensure the following commands: node and npm are available in your console window globally by providing NodeJS and NPM are installed in your PATH.

Once the NodeJS installer has finished, let's test and ensure that NodeJS has been installed and is available globally in our Windows console.

Type node -v into the console and hit Enter. You should see the version number displayed if it has been successfully installed into your PATH; next, try npm -v and see if that also returns a version number.

Installing Bower

Bower is a front-end package manager from the Twitter team and uses Git to consume the packages. To install Bower, we need to install it from the node package manager (npm). In the console window type, the following command: npm install -g bower tells npm to install the Bower package globally. This allows us to use the command Bower in our console window from anywhere. Similar to how Git and NodeJS were added to our PATH.

This is where you will see your console window go nuts as the NPM goes and downloads Bower and its dependencies. Do not panic. You will soon get used to the number of dependencies any NPM package has.

Again, to test that Bower has been installed successfully, use the pavilion -v command.

Bower is not a requirement, but as our front-enders were using it in their workflow, we also needed to install it on our build server.

Installing Grunt

The next thing we need to install and set up for our workflow is GruntJS, the task runner or the equivalent of the front-ender build steps.

To install this, we need to install Grunt from NPM with the following command: npm install -g grunt-cli, which tells the node to install the Grunt command line runner globally and make it accessible from any console window.

To verify it is installed all OK, let's run the obligatory command of grunt -v. This will list a version number and most likely will give you a warning message that it can't find a local grunt in your folder. Don't panic. This is normal as it's trying to look for a gruntfile.js

Installing Ruby

The next part we need to install is Ruby; this allows us to use other packages written explicitly for Ruby. We use Ruby in our setup because our front-enders use a front-end tool called Compass, an extension for the CSS preprocessor Sass.

So we need to download and install the Ruby installer from rubyinstaller.org.

As before, with all installations, we need to ensure that Ruby gets installed correctly and that it gets installed into our PATH.

Once installed, let's verify it is installed correctly by typing ruby -v and gem -v to test it out. As with NodeJS, Ruby has its package manager, and it's called Gem. So we can download packages such as Compass from Gem.

Installing Compass

The final thing to install for our front-end tools is Compass, a tool for the CSS preprocessor Sass, which helps give Sass some extra power with some helpers and extensions.

To install Compass, we must run the following commands in the console.

gem --update system followed by gem install sass and finally gem install compass

We can then do our usual version check and run sass -v and then compass -v

For Compass to be installed, we need to ensure Sass is installed first. Hence, the order above for the commands is important.

Ok, we have everything all set up and ready for the front-end workflow that our front-enders are using; the last thing to set up is the TeamCity plugin to use NodeJS and Grunt in the build steps.

Installing the NodeJS TeamCity Plugin.

The next part of this workflow is downloading and installing the NodeJS TeamCity plugin by Eugene Petrenko from Germany. This plugin is open source and posted on GitHub here - https://github.com/jonnyzzz/TeamCity.Node

However, rather than going over to the GitHub repo, you need to head over to JetBrain's own TeamCity, which hosts this project.

http://teamcity.jetbrains.com/viewType.html?buildTypeId=bt434

This URL will prompt you to log in; do not panic. Click the link to log in as a guest under the login box. Here, you will see the most recent builds for the TeamCity NodeJS plugin. To download the plugin, click the view artefacts button next to the most recent build and download the jonnyzzz.node.zip file. At the time of this post, the most current version is 1.0.46

To install the plugin, you must stop the build runner and the leading web interface services on the server. Once they are arrested, you need to copy the downloaded zip file to TeamCityDataDirectory/plugins; note you do not need to unzip the file; copy it to the plugins folder once the plugin has been copied, restart the two services for TeamCity backup on the build server.

If you were like me and were unsure where the TeamCityDataDirectory is, go to the administration settings page in TeamCity to point you to the folder location path.

Now that you have the plugin installed and have restarted TeamCity go to settings followed by plugin list, which should show the NodeJS plugin in the list if it's installed correctly.

Adding the Build Steps

Each call/command line is its step, so we can easily see where the build fails to help us debug the problem. Each step we add here outputs information, allowing us to keep an eye at all times on what is happening with our build.

Build Log

So when the build is running on TeamCity, we can see the build log and the output of each of these steps to give us an insight into what is happening and if any errors are occurring. The order of these build steps is essential to run in the specific order as defined above. This ensures that if one step fails, it stops, and the other cannot continue running.

"Gotchas"

A few "gotchas" caught me, which I'd like to share with you, which may also see you on your journey to front-end workflow nirvana.

Bower uses Git to go off and consume packages from Git repositories nine times out of ten. It would work fine. However, there may be one dependency it was trying to download where it would fail with a weird error message of 'ECMDERR Failed to execute'. After some Googling, I found the solution was best to change how Git fetches a repo's contents. So, in the console window, please run the following command to ensure it uses the https:// protocol as opposed to git:// URLs, and this error goes away.

git config --global url."https://".insteadOf git://

The second "gotcha" happened when I received some strange errors from NPM. It took a while for me to figure out why I was experiencing these errors. However, our solution was to run the TeamCity service on the server under the Administrator user specifically. After doing so, the errors disappeared, and we managed to get the Grunt workflow all sorted out for our front-enders.

The End…

Hopefully, this has been useful to you, and now you, too, can get up and running with integrating Grunt and the world of NodeJS into your TeamCity Continuous Build processes.

Now front-end Developers and .NET developers can live happily together and stop arguing with one another (maybe!).

I hope this helps you.

Warren :)

Yearning for more of our topics? Check out Cogworks' latest insights on LinkedIn, and feel free to say hey.