Problem:

 I want to read the connection strings from my app.config of my Web Test Project but it never loads.

 Solution:

 Use the Configuration Manager OpenMappedExeConfiguration call to load the app.config file.

 Code: 

// Map the new configuration file.
ExeConfigurationFileMap configFileMap = new ExeConfigurationFileMap();
configFileMap.ExeConfigFilename =
   System.Reflection.Assembly.GetExecutingAssembly().ManifestModule.Name + ".config";

// Get the mapped configuration file
var config = ConfigurationManager.OpenMappedExeConfiguration(
   configFileMap, ConfigurationUserLevel.None);

Explanation:

The app.config of the test project is not loaded for web or load tests, because they are run in the same application domain as the test process (either vstesthost.exe or qtagent.exe), so they will load their config files instead. Therefore, we simply load the configuration file ourselves.

Problem:

I have a data driven web test that calls other web test. The called web tests are also data bound to data loaded by the parent data source.  When I run my test I was getting the following error:

Request failed: Context parameter 'FluidManagement.FluidManagement_json#csv.spacer2' not found in test context

Solution:

Expand the Data Sources node of your web test until you can select the desired table.  From the Properties window change the Select Columns from “Only select bound columns” to “Select all columns”.

Explanation:

The default behavior is to only select the columns that are bound the web test that defines the data source.  This is a reasonable assumption as long as the test does not call any other data bound web test.  If the test you call rely on columns that are not bound in the caller you must change the Select Columns setting.

Problem:

The only way I can verify the success of my web test is to read a value from a database.

Solution:

Create a custom validation rule that can validate the value in the database.

Code:

using System;
using System.ComponentModel;
using System.Data.SqlClient;
using Microsoft.VisualStudio.TestTools.WebTesting;
 
namespace TestUtil
{
   [DisplayName("SQL Validation Rule")]
   [Description("Executes the query and compares the first column of the
    first row to the Expected value.")]
   public class SQLValidationRule : ValidationRule
   {
      public SQLValidationRule()
      {
         IgnoreCase = true;
      }
 
      [Description("The query to execute and extract the first column from.
       In Select * from x where column={0}. You can leave the where clause
       if it is not needed.")]
      public string Query { get; set; }
 
      [DisplayName("Connection String")]
      [Description("The full connection string to the database")]
      public string ConnectionString { get; set; }
 
      [DisplayName("Where Clause Context Parameter Name")]
      [Description("The name of an optional context parameter to use if
       there is a where clause in the query.")]
      public string WhereClauseContextParameterName { get; set; }
 
      [DefaultValue(true)]
      [DisplayName("Ignore Case")]
      [Description("When set to true the case of the word is not used")]
      public bool IgnoreCase { get; set; }
 
      [DisplayName("Expected Value")]
      [Description("The value to compare the first column too.")]
      public string ExpectedValue { get; set; }
 
      public override void Validate(object sender, ValidationEventArgs e)
      {
         string where = null;
         if(!string.IsNullOrEmpty(WhereClauseContextParameterName))
            where = e.WebTest.Context[WhereClauseContextParameterName].ToString();
 
         string result = ExecuteQuery(ConnectionString, Query, where);
 
         e.IsValid = string.Compare(result, ExpectedValue, IgnoreCase) == 0;
      }
 
      private string ExecuteQuery(string connectionString, string query, string where)
      {
         SqlConnection conn = new SqlConnection(connectionString);
 
         string cmdText = string.Format(query, where);
         SqlCommand cmd = new SqlCommand(cmdText, conn);
 
         try
         {
            conn.Open();
            SqlDataReader dr = cmd.ExecuteReader();
 
            if(dr.Read())
               return dr.GetValue(0).ToString();
         }
         catch(Exception e)
         {
            System.Diagnostics.Debug.WriteLine(e.Message);
         }
         finally
         {
            conn.Close();
         }
 
         return null;
      }
   }
}

Explanation:

Creating a custom validation rule for web test is extremely simple.  Simply create a new public class that derives from Microsoft.VisualStudio.TestTools.WebTesting.ValidationRule and override the Validate method.  If you create the class in your test project it will become immediately available the next time you try to add a validation rule to a web request.  If you created the class in a separate class library simply add a reference to that class library in your test project.

This particular validation rule has the five following properties:

1. Query – This is the query to be executed on the database connection.  Only the first column of the first record is used in the comparison of this validation rule.  The query can have a single where condition in the where clause that uses a context parameter value.  For example “Select Name from Table1 where ID={0}”.  At runtime the validation rule will look up the value of the provided context parameter and replace {0} with the value stored in the context parameter.  Using a context parameter is completely optional.
2. Connection String – The connection string to a SQL Server database. This can also be a context parameter entered in {{ContextParameterName}} format.  Otherwise you may simple enter a literal string.
3. Where Clause Context Parameter Name – The context parameter to be used to replace the {0} if any of the query.
4. Ignore Case – Identifies if case should be ignored or not during the string comparison.
5. Expected Value – The value to compare the first column of the first row too.  This can also be a context parameter entered in {{ContextParameterName}} format.  Otherwise you may simple enter a literal string.

You can download the file below.

SQLValidationRule.cs (2.48 kb)

Problem

I can’t easily tell what my Find Text validation rule is searching for in my Web Test.

Solution

Change the DisplayName value in the Web Test xml to something more meaningful.

From Solution Explorer right click on the Web Test and select Open With… then select XML (Text) Editor and click OK.  Once the file is open search the file for DisplayName="Find Text".  Replace Find Text with something more meaningful.  Save the file and close it.  Now right click on the Web Test from Solution Explorer and select Open With… and select the Web Test Editor (Default).  Now expand the Validation Rules folder under the desired web request.  The Find Text validation rule label will be the value you typed in.

Explanation

For some reason the creator of this validation run did not expose the DisplayName property so it could be change from the Properties Window.  However, the value is there and can be changed from the xml file.  This is very helpful when you have several Find Text validation rules on the same request.

Problem:

I have a portion of a web test I only want to run if a specific condition is true.

Solution:

Use the new Visual Studio 2010 Conditional Rule Feature.  However, the condition I wanted to test for was unable to be tested using the out of the box conditions. So in this posting I am going to show you how to write your own custom Condition Rule for Visual Studio 2010.

Code:

using System;
using System.ComponentModel;
using System.Text.RegularExpressions; 

namespace DLB
{
   [DisplayName("Find Text")]
   [Description("Searches the last response for the existence of the specified text.")]
   public class FindTextConditionalRule : Microsoft.VisualStudio.TestTools.WebTesting.ConditionalRule
   {
      [DisplayName("Find Text")]
      [Description("The text to search for in the last response of the web test.")]
      public string Value { get; set; } 

      [DisplayName("Ignore Case")]
      [Description("If true case is ignored while seaching for the string in the response.")]
      [DefaultValue(true)]
      public bool IgnoreCase { get; set; } 

      [DisplayName("Use Regular Expression")]
      [Description("Use a regular expression during search.")]
      public bool UseRegularExpression { get; set; } 

      [DisplayName("Pass If Text Found")]
      [Description("Condition passes if the text is found when this property is set true, or when the text is not found and this property is set false.")]
      [DefaultValue(true)]
      public bool PassIfNotFound { get; set; } 

      /// <summary>
      /// Determines whether the condition was met or not.
      /// </summary>
      /// <param name="sender">The sender of the event.</param>
      /// <param name="e">The Microsoft.VisualStudio.TestTools.WebTesting.ConditionalEventArgs that contains the event data.</param>
      public override void CheckCondition(object sender, Microsoft.VisualStudio.TestTools.WebTesting.ConditionalEventArgs e)
      {
         if(UseRegularExpression)
         {
            Match result = Regex.Match(e.WebTest.LastResponse.BodyString, Value, IgnoreCase ? RegexOptions.IgnoreCase : RegexOptions.None);
            e.IsMet = result.Success;
         }
         else
         {
            int index = e.WebTest.LastResponse.BodyString.IndexOf(Value, IgnoreCase ? StringComparison.CurrentCultureIgnoreCase : StringComparison.CurrentCulture);
            e.IsMet = index == -1 ? false : true;
         } 

         if(PassIfNotFound)
            e.IsMet = !e.IsMet;
      }
   }
}

Explanation:

To create a custom Conditional Rule simply create a public class that derives from Microsoft.VisualStudio.TestTools.WebTesting.ConditionalRule and override CheckCondition. This class can be added directly to your test project or a separate class library referenced by your test project.

From within the CheckCondition method set e.IsMet to true or false depending on if the condition was met. If e.IsMet is set to true all the request under this condition will be executed, otherwise, they will be skipped.

Once your test project has a reference to the conditional rule class it will be shown in the Add Conditional Rule and Items to Condition dialog.

Once the condition is added to your web test you will see the If statement with the selected requested nested underneath.

The sample code is modeled after the Find Text Validation rule and exposes many of the same properties. The purpose of this condition is to search the last response of the web test for the existence or nonexistence of the specified text. If the condition is met the requests under this condition will execute, otherwise the requests will be skipped.

You can download the sample file from here.

FindTextConditionalRule.cs (2.11 kb)

Problem:

I need to apply the same validation rule to every request of my web test to test for forbidden text.

Solution:

Create a web test plug-in to insert the validation rule before each request.

Code:

using System;
using System.ComponentModel;
using Microsoft.VisualStudio.TestTools.WebTesting;
using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

namespace DLBRacingTest.Custom
{   
   [DisplayNameAttribute("Forbidden Text")]   
   [Description("Adds a validation rule on every request looking for forbidden text.")]   
   public class ForbiddenText : WebTestPlugin   
   {       
      /// <summary>       
      /// Optional , delimited list of forbidden text. If left blank 'Page Error' is the only       
      /// forbidden text.       
      /// </summary>       
      public string Values { get; set; }         
      public override void PreRequest(object sender, PreRequestEventArgs e)        
      {      
         base.PreRequest(sender, e);

         if (string.IsNullOrEmpty(Values))                
            Values = "Page Error";             

         foreach (string value in Values.Split(",".ToCharArray()))            
         {                
            ValidationRuleFindText pageErrorRule = new ValidationRuleFindText();                
            pageErrorRule.FindText = value;                
            pageErrorRule.IgnoreCase = true;                
            pageErrorRule.UseRegularExpression = false;                
            pageErrorRule.PassIfTextFound = false;                
            e.Request.ValidateResponse += new EventHandler<ValidationEventArgs>(pageErrorRule.Validate);            
         }        
      }   
   }
} 

Explanation:

While running test on my web site I noticed a test passed although one of the pages had a Page Error on it. The reason the test passed was because there were no required extraction rules, no validation rules and the paged it failed on was the intended page. However, this test obviously should have failed. The existence of the word “Error” on any of my sites pages is a failed test. Something has gone wrong and I need to be alerted.  By this point I have hundreds of web test each with many request. The thought of having to add a validation rule to every request of every test was very discouraging.

I just knew there had to be an easy way to add this validation rule to every request without doing it explicitly.  A web test plug-in actually has callback that is called before every request of your test called PreRequest.  So I simply add the validation rule when this callback is called.  Now I only have to add a single web test plug-in and every request of that test now has a validation rule that will fail if any forbidden text is found.

I added an optional parameter to the web test plug-in called Values. If you leave it blank a single validation rule is added that searches for the text “Page Error” and fails if it is found.  If you have other forbidden words you would like to search for simply set Values to a comma delimited list of values (i.e. “Page Error,Invalid”).

To use this web test plug-in you can either create a separate class library and reference it in your test project, or simply add this class to your test project. Once you do and build the code you can right click on the root of your web test and select Add Web Test Plug-in… from the context menu.  Select “Forbidden Text” from the list and click OK.

Problem:

I need dates in my WebTest to always be in the future.

Solution:

Create a WebTest Plug-in to generate the dates and set Context Parameters to hold the values.

Code:

using System;
using System.ComponentModel;
using Microsoft.VisualStudio.TestTools.WebTesting;

namespace DLBRacingTest.Custom
{
   [Description("Stores four dates in the test context [ContextParameterName], [ContextParameterName]Month, [ContextParameterName]Year, [ContextParameterName]Day")]
   public class GenerateDates : WebTestPlugin
   {

      [Description("The number of days to add.")]
      public double Days { get; set; }

      [Description("The number of Hours to add.")]
      public double Hours { get; set; }

      public string ContextParameterName { get; set; }

      public override void PreWebTest(object sender, PreWebTestEventArgs e)
      {
         DateTime temp = DateTime.Now.AddDays(Days).AddHours(Hours);
         e.WebTest.Context.Add(this.ContextParameterName, temp.ToString("ddd, MMM d, yyyy"));
         e.WebTest.Context.Add(this.ContextParameterName + "Month", temp.Month.ToString());
         e.WebTest.Context.Add(this.ContextParameterName + "Year", temp.Year.ToString());
         e.WebTest.Context.Add(this.ContextParameterName + "Day", temp.Day.ToString());
      }
   }
}

Explanation:

Add the code above to any test project or create a class library and reference the library from your test project.  Doing so will make this WebTest Plug-in available to be added to your web test.

To add the WebTest Plug-in to your test simply right click root of your web test and select “Add WebTest Plug-in...”  You will be shown a dialog to select the desired plug-in.

There are three parameters with this plug-in. The first two are days and hours to be added to the current date.  These values can be negative if you want a date in the past or 0 if you want todays date.  The third parameter is the context parameter name you want assinged to the context parameter.  In the screen shoot above we request a date 7 days in the future to be assinged to a context parameter named MyDate.  Note that this plug-in actually creates 4 context parameters as explained in the description area of the plug-in dialog.  If todays date is Jan 20, 2009 the resulting context parameters would be:
MyDate = Tue, Jan 27, 2009
MyDateMonth = 1
MyDateYear = 2009
MyDateDay = 27

Note that unlike Context Variables generated by Extraction Rules the Context Variables added by WebTest Plug-ins do not appear in the Value drop down in the properties window of a Form Post Parameter. There are two methods to assigning the Context Parameters to Form Post Parameters in your web test.  The first method is to simply type the name of the Context Parameter between double open and double close curly braces i.e. {{MyDate}}.  This method is quick and easy but is suspectable to errors if you miss type the variable name.  This method can also become tedious if you have to use this Context Variable in many locations of your web test.

The second method is to add Context Variables (as place holders) to your Web Test.  This will place the Context Variables in the Values drop down in the properties window.   This ensures the correct spelling is used and makes updating the test very easy.  To add a Context Variable simply right click on the root of your web test and select "Add Context Variable" from the context menu.  Select the Context Variable and update the Name property to the name of your context parameter i.e. MyDate.  You can leave the context variables or delete them once you have them bound to all the request in your test.  If you delete them however, click No when asked if you would like to clear the bindings.

Now everytime you execute this web test the dates will be 7 days in the future of the date you execute the test.

I have gotten some great feedback from the early adopters of Neptune and have begun development on a new version.

The most significant new feature is a POP3 interface so you can point Outlook or Outlook Express (or any POP3 Mail Client) at port 110 and see all the messages in Neptune. This will remove the need to open the trace.log file to view the contents of the processed messages. I noticed also while creating the POP3 interface that Outlook 2007 deviates from the SMTP standard by adding a space after ‘FROM:’ in the MAIL command and after the ‘TO:’ in the RCPT commands. Neptune was updated to handle this deviation.

The next idea is to allow Neptune to fail on purpose and return error codes so you can test your exception cases.

Please add comments with other suggestions for the next release.

Thanks for the support and continue to spread the word.

While working on a current project I found myself faced with testing code that sends email.  In the past I would end up with an inbox full of test messages or unhappy customers that wonder why they just received a flood of emails from my site.  I was also frustrated that there was no way to easily automate this testing.

Enter Neptune. Neptune is a SMTP Development Server targeted for use in automated testing. I simply asked the question, “what if I had a SMTP server that did not relay the message and allowed me to query for messages and their content”. I would be able to use a server of this type to act as my SMTP server during testing and write custom plug-ins, validation and extraction rules to communicate with the server.

The goals of Neptune were to facilitate automated testing and be easy to use. I did not want to have to install a service or understand everything there is to know about running a SMTP server. I simply want to start Neptune and run my test.

Full documentation on how to use Neptune is provided in the msi you can download here. NeptuneSetup.msi (1.27 mb)

If you find Neptune usefull feel free to Donate for future development.