How to run xunit test with vNext Build

I was recently asked for help getting xunit test to run using VSO build vNext. In this post I will walk you through how I was able to get everything working.

I created a simple Class Library project and added it to source control. Make sure and choose a Windows - Class Library and not a Web Class Library (Package).

Now using Nuget Package Manager add a reference to xunit and xunit.runner.visualstudio. To the class1.cs file add a using for Xunit and add a test method.

Run the test and makes sure it passes.

Now check in your changes so we can configure the build. Navigate to VSO and the Build Hub. Add a new build using the Visual Studio template.

Update the solution path on the Visual Studio Build step and update the Mappings on the Repository tab. Now return to the Build tab so we can correct the Visual Studio Test step to run our xunit test. The key to getting the xunit test to run is setting the Path to Custom Test Adapters value under the Advanced section. The value has to point to the folder under the Packages folder that contains the xunit test adapter. I used the following path:

$(Build.SourcesDirectory)\Packages

Now simply save and queue your build.

Comments (9) -

  • Sam

    7/8/2015 9:02:09 PM | Reply

    Nice and terse.
    Do you know how to achive the same result with on-prem TFS2012?

    Thanks
    Sam

  • Alan Whittle

    11/30/2015 10:48:15 AM | Reply

    Hi Donovan, I'm trying to use this example for running xUnit tests on VSO but getting the error message ##[error]Could not find file 'C:\a\1\s\packages'.  

    Any ideas what I am missing?

    • Donovan

      1/3/2016 2:48:02 AM | Reply

      Sorry for the delay.  Make sure you set the Path to Custom Test Adapters. donovanbrown.com/.../how-to-run-xunit-test-with-vnext-build

      • Will

        9/2/2016 9:56:27 PM | Reply

        Having the same issue as Alan ,

        Have tried.

        $(Build.SourcesDirectory)\Packages
        $(Build.SourcesDirectory)\MySolutionName\Packages
        $(Build.SourcesDirectory)\MySolutionName\MyTestProjectName\Packages
        $(Build.SourcesDirectory)\MySolutionName\MyTestProjectName\bin\Debug

        Notta !  I've read your article along with 10 others that advise similar approaches.  Is there no way of determining where on Team Services hosted agent that the Nuget packages are being installed to ?  

        logs from the Nuget restore don't show much.  etc.

  • Paul

    2/12/2016 4:06:53 PM | Reply

    Donovan,

    I'm having a problem where I have a VS2013 solution I'm trying to build with VNext. It is a web project and the unit tests are in a separate project, which is proper design. During the build process, when it gets to the test project I get:

    2016-02-12T15:20:04.9595528Z      1>Project "C:\a\1\s\XXX\src\XX.XX.XX\XX.XX.XX.sln" (1) is building "C:\a\1\s\XXX\src\XX.XX.XX\XX.XX.XX.Services.OData\XX.XX.XX.Services.YY.csproj" (4:4) on node 1 (Publish target(s)).
    2016-02-12T15:20:04.9595528Z      4>_DeploymentUnpublishable:
    2016-02-12T15:20:04.9605536Z          Skipping unpublishable project.

    Then, when it reaches the test step I get:

    No test assemblies found matching the pattern: '**\*tests*.dll'

    • Donovan

      2/28/2016 1:37:15 AM | Reply

      Does your test project have the word 'tests' anywhere in the name? If not it will not be found.  The test task searches for any dll that has the word tests in it.  You can change the pattern to fine your dll.

      • Paul

        2/28/2016 5:10:38 PM | Reply

        Donovan,

        The test assemblies were not part of the published project so they don't get pushed to the build server. If I add the test assembly in the References they get published/deployed as part of the build and then they get found and the xunit tests execute. However, this isn't desired. We certainly don't want the test assemblies getting published just so the unit tests can execute during the build process. Is there a workaround in vNext for this problem?

        • Donovan

          3/3/2016 5:13:15 AM | Reply

          When I want to run unit test as part of my build my sln will have both the test projects and the projects to be tested in a single solution.  However, after the build I use the Copy and Publish Build Artifacts task to select just the files I want to publish. This will leave the test dlls behind on my build machine only selecting the files that need to be in my release.
          However, just because you publish files as artifacts does not mean they have to be deployed from the release. There are times I also want to run tests during my release like UI tests. For those tests to be executed during my release the dlls holding those tests must also be published from our build.  The release will deploy the actual application and only use the test dlls to run the UI tests from the agent after the real application has been deployed to the environment.

  • Alejandra

    8/10/2018 4:49:06 AM | Reply

    Hello Donovan,

    can you please help me with this error?

    2018-08-10T04:43:27.9642198Z ##[warning]RunMessage : Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests in C:\Users\Administrador\Desktop\SquadComercial\_work\5\a\ADV.Testing.dll
    2018-08-10T04:43:27.9650696Z ##[warning]RunMessage : The NUnit 3 driver encountered an error while executing reflected code.
    2018-08-10T04:43:27.9651362Z ##[warning]RunMessage :    en NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
    2018-08-10T04:43:27.9651939Z    en NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
    2018-08-10T04:43:27.9652174Z    en NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
    2018-08-10T04:43:27.9652469Z    en NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
    2018-08-10T04:43:27.9652645Z    en NUnit.Engine.Runners.TestDomainRunner.LoadPackage()
    2018-08-10T04:43:27.9652834Z    en NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
    2018-08-10T04:43:27.9653124Z    en NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
    2018-08-10T04:43:27.9653282Z    en NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
    2018-08-10T04:43:27.9653526Z    en NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, TestFilter filter) en D:\repos\nunit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:línea 229
    2018-08-10T04:43:27.9654218Z ##[warning]RunMessage : Innerexception: System.Security.VerificationException: La operación no puede desestabilizar el runtime.

Add comment

Loading