Uploading automated test results to MTM made "easy"

As you may know, Microsoft Test Manager is capable of running both manual and automated tests. This is great because it allows you to manage your testing in one centralized location. The problem is that the setup can be pretty involved and a lot of projects just aren’t willing or don’t prioritized the setup.

So, it there a workaround?

Yes, Visual Studio ships with all the tools you need to run and upload automated tests.

How?

  1. Created an automated test method

  2. Associated the test method with a test case

  3. Add the test case to a suite in MTM


    Note the Configuration and Suite ID as you will need them to upload the results

  4. Run the test
    You can use MSTest or VSTest to run the test and generate a TRX file.
    Note* You can use the TRX created by a TFS build

    Sample MSTest command line:
    MSTest.exe /testcontainer:"C:\DLL\UnitTestProject1.dll" /resultsfile:testResults.trx

    Sample VSTest command line:
    VSTest.console.exe "C:\DLL\UnitTestProject1.dll" /Logger:trx /Settings:"C:\DLL\TestSettings.testsettings"
    Note* You will need to use the testsettings file with VSTest or you will not be able to upload the test results

  5. Use TCM to upload the resulting TRX file
    Note* There is problem with VSTS (formerly VSO) that causes these automated tests to stay in a state of “In Progress”

    Sample TCM command line:
    TCM run /publish /suiteid:ID# /configid:CONFIG# /resultowner:"USER" /resultsfile:"C:\DLL\testResults.trx" /collection:"TFS_COLLECTION_URL" /teamproject:"PROJECT_NAME"

What is the down side?

If you skip the Microsoft recommended setup you will not be able to kick off automated tests from MTM. This is a neat feature, but of limited use. It kind of a pain to use MTM to kick off automation tests and the tests themselves take forever to run.