Publish using MSBuild like VS2010

A few weeks ago I talked about automating the process of updating, compiling and deploying from the source code to running web application. I’ve done some improvements since then; the most relevant was to apply the web.config transformation phase. I admit it, it’s something I’ve read once at a glance and almost forgot it, in part because I didn’t understand very well on the fly. But this I really needed it and made a second review and Voilà! By the way, Great job in the book “Pro ASP.NET MVC 3 Framework” from Apress. All I needed was to invoke MSBuild with the appropriate arguments so it makes exactly the same as in the Publish Dialog from Visual Studio 2010.

After spending some time making Google-research and consequently StackOverflowing a lot, almost everybody agreed that the solution was modify the .csproj or .vbproj and include something like this:

<Target Name="PublishToFileSystem" DependsOnTargets="PipelinePreDeployCopyAllFilesToOneFolder">
    <Error Condition="'$(PublishDestination)'==''" 
 Text="The PublishDestination property must be set to the intended publishing destination." />
        <MakeDir Condition="!Exists($(PublishDestination))" 
  Directories="$(PublishDestination)" />
        <ItemGroup>
            <PublishFiles Include="$(_PackageTempDir)\**\*.*" />
        </ItemGroup>
        <Copy SourceFiles="@(PublishFiles)"
  DestinationFiles="@(PublishFiles->'$(PublishDestination)\%(RecursiveDir)%(Filename)%(Extension)')"
  SkipUnchangedFiles="True" />
</Target>
 

And invoke the MSBuild with /t: PublishToFileSystem and include in the property parameter /p: PublishDestination with the location where to put the final compiled files. Everything looks great? But there is small problem, I don’t want to modify the .csproj file, I just want to do exactly what VS2010 does when you right-click and select Publish, as I said before. I want my script to be reusable and do not have to remember in each project to include obscure XML fragments. So the quest begins.

I started to learn a little about MSBuild syntax and quickly went to the file Microsoft.WebApplication.targets, which is located very deep in MSBuild Folder but easily located by opening any .csproj file. Well, what’s inside this odd file? This file defines all target used by VS2010 when compiling or deploying a project. Since the new target defined by stackoverflow.com people depends on PipelinePreDeployCopyAllFilesToOneFolder target, I went to hunt it and I discovered vital information about the parameters that target use for to copy the files. I started the experimentation phase, and finally I lined up the stars and Voilà encore une fois! The magic combination is as follows:

The MSBuild receives a positional parameter with the .csproj, no change with it. The target as commented before is /t:PipelinePreDeployCopyAllFilesToOneFolder, the properties are /p:Configuration=Release; BuildInParallel=true; PackageAsSingleFile=False; AutoParameterizationWebConfigConnectionStrings=false. All this for the following: build in release mode, optimal settings for production server, take advantages of multi core processing, do not package as a .zip file the result and do not set replaceable garbage in web.config, this is not necessary because I don’t want to import the result with IIS wizard. Another important property is /p:IntermediateOutputPath=..\TempObjWeb\ this is specified to avoid a mysterious compilation error after a deploy, it’s something related with the web.config remains inside the project file and it can’t be parsed because it’s not inside a folder configured as virtual directory or application in IIS. The most important property is /p:_PackageTempDir= compilation target, that’s I’ve found following the trace from target to dependent target inside the Microsoft.WebApplication.targets file, when setting this property, the destination path (like the dialog in VS2010) for the compiled web elements.

Other improvements I’ve done in my script are at clean phase, this time all the temporary folders are deleted totally before the compilation begins. Continuing with the delete elements, when I copy the final compilation results (using aspnet_compiler) this time I make a smart delete excluding vital folders for specific use in my application, using the -Exclude in powershell. I also added some new parameters to promote its maintenance. I think there are still too many things to do but as I am still learning I continuously improve my toolbox. I let you the entire script code.

$aspnetcompiler = $env:SystemRoot + "\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe"
$msbuild = $env:SystemRoot + "\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe"
$repo = "C:\Users\Administrator\Desktop\livost-repo"
$webapp = $repo + "\LivostWeb"
$webtarget = $repo + "\LivostWebPublish"
$fullcomptarget = $repo + "\WebCompiled"
$comptarget = "..\WebCompiled\"
$sitename = "IIS:\Sites\mvclivost"
$urltest = "http://localhost:8000/Home"
#$destitny = "Debug"
$destitny = "Release"

import-module .\PScommon\psake\psake.psm1
import-module .\PSCommon\WebAdministration
Set-Alias -Name ipsake -Value Invoke-psake

$old_pwd = pwd
cd $repo
hg pull
hg update

$msbuild_arg0 = $repo + "\Livost.sln"
$msbuild_arg1 = "/p:Configuration=$destitny;BuildInParallel=true"
$msbuild_arg2 = "/t:Clean"
$msbuild_arg3 = "/m"
$msbuild_arg4 = "/v:m"
$msbuild_arg5 = "/nologo"
$msbuild_arg6 = "/clp:Verbosity=minimal"
$msbuild_args = @($msbuild_arg0, $msbuild_arg1, $msbuild_arg2, $msbuild_arg3, 
$msbuild_arg4, $msbuild_arg5, $msbuild_arg6)

Write-Host "Cleaning the solution"
Write-Host "Executing $msbuild $msbuild_args"
& $msbuild $msbuild_args > out.txt

# removing temporary folders
$cleanwebtarget = $webtarget + "\*"
rm $cleanwebtarget -Force -Recurse
$cleanfullcomptarget = $fullcomptarget + "\*"
rm $cleanfullcomptarget -Force -Recurse
$cleantempobj = $repo + "\TempObjWeb\*"
rm $cleantempobj -Force -Recurse

# keep the rebuild step 
$msbuild_arg2 = "/t:Rebuild"
$msbuild_args = @($msbuild_arg0, $msbuild_arg1, $msbuild_arg2, $msbuild_arg3, 
$msbuild_arg4, $msbuild_arg5, $msbuild_arg6)

Write-Host "Rebluilding the solution"
Write-Host "Executing $msbuild $msbuild_args"
& $msbuild $msbuild_args >> out.txt


# modified here (publish instead of rebuild) this change allows to apply the 
web.config transformation 
$msbuild_arg0 = '"' + $webapp + '\LivostWeb.csproj"'
$msbuild_arg1 = "/t:PipelinePreDeployCopyAllFilesToOneFolder"
$msbuild_arg2 = "/p:Configuration=$destitny;BuildInParallel=true;
PackageAsSingleFile=False;
AutoParameterizationWebConfigConnectionStrings=false"
$msbuild_arg3 = "/p:IntermediateOutputPath=..\TempObjWeb\"
$msbuild_arg4 = "/p:_PackageTempDir=$comptarget"
$msbuild_arg5 = "/nologo"
$msbuild_arg6 = "/clp:Verbosity=minimal"
$msbuild_args = @($msbuild_arg0, $msbuild_arg1, $msbuild_arg2, $msbuild_arg3, 
$msbuild_arg4, $msbuild_arg5, $msbuild_arg6)

Write-Host "Building the Web Application"
Write-Host "Executing $msbuild $msbuild_args"
& $msbuild $msbuild_args >> out.txt

$anc_arg0 = "-v"
$anc_arg1 = "/"
$anc_arg2 = "-p"
$anc_arg3 = $fullcomptarget
$anc_arg4 = "-f"
$anc_arg5 = $webtarget
$anc_arg6 = "-c"

$asncargs = @($anc_arg0, $anc_arg1, $anc_arg2, $anc_arg3, $anc_arg4, $anc_arg5, 
$anc_arg6)

if (-not (Test-Path $webtarget)) {
    mkdir $webtarget > $null
}

Write-Host "Precompiling web application"
Write-Host "Executing $aspnetcompiler $asncargs"
& $aspnetcompiler $asncargs >> out.txt

$webSite = Get-Item $sitename

$poolName = $webSite.applicationPool
$pool = Get-Item "IIS:\AppPools\$poolName"
    
if ((Get-WebAppPoolState -Name $poolName).Value -ne "Stopped") {
    Write-Host "Stopping the Application Pool"
    $pool.Stop()
    Start-Sleep 3
}

Write-Host "Smart delete..."
$todelete = $webSite.physicalPath + "\*"
rm $todelete -Force -Recurse -Exclude @("Files","_temp_upload", "App_Data")

Write-Host "Copying files..."
$source = $webtarget + "\*"
cp $source $webSite.physicalPath -Force -Recurse

Write-Host "Waiting a few seconds..."
Start-Sleep 5
Write-Host "Starting the Application Pool"
$pool.Start()

& "${env:ProgramFiles(x86)}\Internet Explorer\iexplore.exe" $urltest

cd $old_pwd
 

Automate source code to running web process on testing server

For those who develop web applications and share code with others developers, the process of keep up to date the test server may be a little tedious and boring, even if you have the server at one click RDP distance. Well, as an aside, I also have it, but with a slight difference in bandwidth (I use a dialup connection @ 56Kbps and the RDP window is 800x600), and believe me, this can be annoying.

The process consists in the following steps, when we (the developers) want to make a test in “production” scenario the first is commit all changes to the repository. I am using Mercurial as VCS, surprise?! I’ve set up as an IIS website following the guide found in http://stackingcode.com/blog/2011/02/24/running-a-mercurial-server-on-iis-7-5-windows-server-2008-r2 and it helped me a lot, I recommend it for the HG fans. Let’s continue, after commit all changes to the repository on the server, we’ve to pull and update the local copy there in the server. Then make a clean and rebuild the solution, and the real deploy with aspnet_compiler for to have a folder with the content fully compiled according to MSDN. The deploy continues with the IIS, go to Application Pools stop the selected Pool, copy the files to the destination folder and finally start the Pool again. Optionally you can start a web browser with the url for verify everything is ok.

As you can see there are several steps involved, and several clicks as well (remember, for me, every click in the server is like a needle ticking in the arm) how to resolve it? There are two requirements for use the solution I propose: the hg executable must be in Windows PATH and the other thing I’ve done I don’t know if It’s really necessary but it worked to me, Replace the content of “C:\Program Files (x86)\MSBuild” folder from a machine with Visual Studio installed to the server (there was something about Microsoft.target not found) I admit it, I didn’t dig in that problem, I someone can understand why that happens, comments will be welcome.

After quickly analysis of ways to do it, I decided to use PowerShell as script engine, due to its flexibility and high integration level with .NET and any Windows component.  This is how my script is done:

$aspnetcompiler = $env:SystemRoot + "\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe"

$msbuild = $env:SystemRoot + "\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe"

$repo = "C:\Users\Administrator\Desktop\livost-repo"

$webapp = $repo + "\LivostWeb"

$webtarget = $repo + "\LivostWebPublish"

$sitename = "IIS:\Sites\Default Web Site\mvclivost"

$urltest = "http://localhost/mvclivost/Home"

These are some variables that may change between servers and projects, such as the MSBuild and aspnet_compiler location, the folder where I have the repository, web and target folders, site name.

import-module .\PScommon\psake\psake.psm1

import-module .\PSCommon\WebAdministration

Set-Alias -Name ipsake -Value Invoke-psake

These imports are taken from this guide that explains how to interact with IIS from powershell http://www.yangq.org/2011/04/09/automate-asp-net-deployment-with-powershell-install-and-update

$old_pwd = pwd

cd $repo

hg pull

hg update

Here I save the original Working directory and move to local repository directory, then pull and update, really easy.

$msbuild_arg0 = $repo + "\Livost.sln"

$msbuild_arg1 = "/p:Configuration=Release;BuildInParallel=true"

$msbuild_arg2 = "/t:Clean"

$msbuild_arg3 = "/m"

$msbuild_arg4 = "/v:m"

$msbuild_arg5 = "/nologo"

$msbuild_arg6 = "/clp:PerformanceSummary;Verbosity=minimal"

$msbuild_args = @($msbuild_arg0, $msbuild_arg1, $msbuild_arg2, $msbuild_arg3, $msbuild_arg4, $msbuild_arg5, $msbuild_arg6)

Write-Host "Cleaning the solution"

Write-Host "Executing $msbuild $msbuild_args"

& $msbuild $msbuild_args > out.txt

Here I set up the parameters for clean and write to the output properly what’s going on and to a file the text waterfall with all msbuild details.

$msbuild_arg2 = "/t:Rebuild"

$msbuild_args = @($msbuild_arg0, $msbuild_arg1, $msbuild_arg2, $msbuild_arg3, $msbuild_arg4, $msbuild_arg5, $msbuild_arg6)

Write-Host "Rebluilding the solution"

Write-Host "Executing $msbuild $msbuild_args"

& $msbuild $msbuild_args >> out.txt

The same for rebuild the solution after to have cleaned up old compiled files.

$anc_arg0 = "-v"

$anc_arg1 = "/"

$anc_arg2 = "-p"

$anc_arg3 = $webapp

$anc_arg4 = "-f"

$anc_arg5 = $webtarget

$anc_arg6 = "-c"

$asncargs = @($anc_arg0, $anc_arg1, $anc_arg2, $anc_arg3, $anc_arg4, $anc_arg5, $anc_arg6)

if (-not (Test-Path $webtarget)) {

    mkdir $webtarget > $null

}

Write-Host "Precompiling web application"

Write-Host "Executing $aspnetcompiler $asncargs"

& $aspnetcompiler $asncargs >> out.txt

Setting up the parameters for aspnet_compiler, for me this was the hardest part; there are so many arguments combinations to try! But finally I got it, I also ensure that the target folder exists before proceed with precompilation.

$webSite = Get-Item $sitename

$poolName = $webSite.applicationPool

$pool = Get-Item "IIS:\AppPools\$poolName"

   

if ((Get-WebAppPoolState -Name $poolName).Value -ne "Stopped") {

    Write-Host "Stopping the Application Pool"

    $pool.Stop()

}

Write-Host "Copying files..."

$source = $webtarget + "\*"

cp $source $webSite.physicalPath -Force -Recurse

Write-Host "Waiting a few seconds..."

Start-Sleep 5

Write-Host "Starting the Application Pool"

$pool.Start()

As I mentioned before I’ve taken from that fellow’s blog, here is when I interact with IIS, first I grab the AppPool object by its name and if it isn’t stopped then stop it. After that copy everything to the target physicalpath and wait about 5 seconds, why? Sometimes these steps may be executed too fast and the services stay in an “irregular state” such as starting-up or stopping and at this stage the services control doesn’t receive control messages, such as start or stop, at least this is what Microsoft says in the documentation for the error I receive sometimes if I don’t wait these few seconds.

& "${env:ProgramFiles(x86)}\Internet Explorer\iexplore.exe" $urltest

cd $old_pwd

And finally start a new instance of Internet Explorer for verify everything is ok and the ASP.NET makes its first and long runtime compilation with me and any other bad lucky guy.

Now, how to use it, in the server must be allowed the scripts execution, this can be done using a powershell console as administrator, then type ‘Set-Executionpolicy remotesigned’, in the example I’ve put a run.cmd with this text ‘powershell -command ./automate.ps1’ where this informs to powershell to execute the script. Important this script must be run with administrator privileges because of its interaction with IIS. Maybe you have to modify the content of run.cmd and instead of ./automate.ps1 to make a cd to the folder where the script is and then execute it as before. This occurs because when you execute a cmd as administrator the current directory is changed automatically and mysteriously to C:\Windows\system32. I hope this post to be useful to you, till the next!