Click here to monitor SSC
  • Av rating:
  • Total votes: 58
  • Total comments: 15
Damon Armstrong

Token Replacement in ASP.NET

01 November 2006

As a web developer, you will encounter situations that call for an effective token replacement scheme. Token replacement is really just a technologically-savvy way of saying string replacement, and involves substituting a "real" value for a "token" value in a string. This article presents a powerful approach to token replacement in ASP.NET 2.0.

The goal is to create a centralized token replacement mechanism that works in ASP.NET controls, HTML controls, static HTML markup, and in text placed on the page from the code-behind. In other words, in any situation imaginable.

If you want to skip the background info and get right into the replacement mechanism, then jump to the section titled Acquiring All ASP.NET Page Content. Otherwise, let's get some background on the various token replacement options in ASP.NET.

Basic token replacement concepts

The ultimate goal of token replacement is to make the value of a string dynamic. You begin with a static string containing tokens, and then update those tokens with replacement values to produce your dynamic string. For example, you may have the following string in an application that emails users a new password if theirs has been lost:

Dear [$NAME$],

You recently requested a password reset.  Your new password is [$PASSWORD$].
Please keep this password in a safe location and quit losing it.

Thank You 

In this scenario, the application has two values to communicate to the user: their name (to personalize the email) and their new password (because they need it to login). Let's say we wish to assign the value "Matt" to the [$NAME$] token and "5ZQS76Bv" to the [$PASSWORD$] token. The resulting text would look like this:

Dear Matt,
You recently requested a password reset.  Your new password is 5ZQS76Bv.
Please keep this password in a safe location and quit losing it.
Thank You

One question that might arise is, why not just use concatenation to build the string? Concatenation is certainly faster, but it comes down to a question of maintainability. Concatenation can only occur in code, so you would have to hard-code the majority of the string, like this:

"Dear " + Name + ",\r\nYou recently requested a password reset.
Your new password is " + password + ". Please keep this password in a
safe location and quit losing it.\r\n\r\nThank You"

What happens if you wanted to update the email content? You would have to update your code, recompile it, and redeploy it. Plus, a complex HTML email built directly in C# source is not the easiest thing to debug and maintain.

The token replacement approach allows you to store the content in a separate file, read it into your application, and replace the tokens in code. This separates the content from the application and allows you to make updates in a less convoluted environment and without having to recompile or redeploy the application.

Token / String Replacement in ASP.NET

There are several ways to replace strings in ASP.NET 2.0, but I'm only going to touch on the three that I consider the most popular:

  • Inline Script
  • The String.Format method
  • The String.Replace and StringBuilder.Replace instance methods

Each of these are outlined in more detail below.

Inline Script

Although ASP.NET has moved to a code-behind model, you can still write inline script just as you could in original ASP. Inline script still has its uses and can be extremely helpful for string replacement scenarios. If you think about it, inline script is essentially a really powerful string replacement mechanism that finds your script in a string (i.e. the page markup), runs that script, and then replaces the script with the value it produces. Here's our original example modified for inline script:

Dear <%=Name%>,
You recently requested a password reset.  Your new password is <%=Password%>.
Please keep this password in a safe location and quit losing it.
Thank You

One issue with inline script is that you cannot always use it in conjunction with ASP.NET controls. For example, let's say that you wanted a standard ASP.NET button on the page to say "Matt, Click Here to Email Your Password." You cannot do the following:

<asp:Button runat="server" id="btnPwd"
     text="<%=Name%>, Click Here to Email Your Password" />

Nor can you use the mechanism from inside your code-behind. So, it's useful for making a single page, but not for our global string-replacement needs.

The String.Format Method

The String.Format method accepts a string containing tokens followed by a list of replacement values for the tokens found in the string. The tokens in the string are numbers surrounded by curly brackets, where the number corresponds to the index of the replacement value passed into the method.

String.Format("Name: {0}, Gender: {1}, Age:{2}, Height:{3}, Weight:{4}",
    "Matt", "Male", "25", "5'10\"", "160");

//OUTPUT-->Name: Matt, Gender: Male, Age: 25, Height: 5'10", Weight: 160

The first token, {0}, corresponds to the first replacement value "Matt", the second token, {1}, corresponds to "Male", and so on. You can have an infinite number of tokens and replacement values, but you have to have at least as many replacement values as you have tokens. In other words, when the function encounters a number in curly brackets, {N}, there had better be a corresponding replacement value for that token or else the function throws an exception.

// Throws an exception because there are 4 tokens
// but only one replacement value

String.Format("Name: {0}, Gender: {1}, Age:{2}, Height:{3}, Weight:{4}",
    "Matt");

Tokens do not need to appear in order, you can use a token more than once in the string, and you can have more replacement values than tokens:

String.Format("{1}{3}{2}{2}{0}",
    "O","H","L","E","X","Y","Z","!");

//OUTPUT-->HELLO

In this example, the token, {2}, appears twice, the tokens are not in order, and the "X", "Y", "Z", and "!" replacement values are never used.

You can easily put tokens for the string.Format method in ASP.NET controls, HTML controls, static HTML markup, and code in the code-behind, so it is a candidate for use in our global string replacement mechanism. My biggest issue is that the tokens are non-intuitive. When you see a token like [$NAME$], you have some idea what it represents, whereas the token, {7}, communicates very little.

The String.Replace and StringBuilder.Replace Methods

Lastly, we have the Replace methods found on the String and StringBuilder instances. Both of these methods operate using the same basic logic. You provide a string containing a token, identify the token, supply a replacement value for the token, and the method replaces any instances of the token with the replacement value.

We'll begin by looking at the Replace method on a String instance. Since the Replace method is only available on a String instance and not from the String class itself, you start by creating a String instance. Then you call the Replace method from the instance by passing in a token and replacement value, and the method returns a string containing the replacements. Your original string, however, remains unchanged. If you want to update your string with the replacement, you have to assign the result of the Replace function back to your string, as demonstrated in the following example:

string myString = "Hello, my name is [$NAME$].";

//This does NOT change the value of myString
myString.Replace("[$NAME$]", "Matt");

//You have to assign the value of the Replace function back
//to the string to change the value
myString = myString.Replace("[$NAME$]", "Matt");

You call the Replace method on a StringBuilder instance in the exact same way as a string: by passing in a token and replacement value. However, the Replace method on a StringBuilder instance operates directly on that StringBuilder instance's value, so you do not need to assign the result of the Replace function back to your StringBuilder to pick up the changes:

StringBuilder myBuilder = new StringBuilder("Hello, my name is [$NAME$].");
myBuilder.Replace("[$NAME]","Matt");

Tokens like [$NAME$] are fairly intuitive, can be placed in ASP.NET controls, HTML controls, static HTML markup, and code in your code behind. So it's the option we're going to run with for building the global string replacement mechanism for our ASP.NET application.

Acquiring All ASP.NET Page Content

Our biggest objective in building a global string replacement mechanism is that it needs to work everywhere. If you put a token directly in your HTML, it needs to be replaced. If you assign a token to an ASP.NET control from a code-behind, then it needs to be replaced. If you put a token in a database and a control pulls that value from the database and displays it, then it needs to be replaced. So… how do you go about doing that?

When it comes right down to it, all of the architecture and code for ASP.NET revolves around building a giant string to send to the browser. ASP.NET controls, HTML, code, values from a database, they all end up as part of a string containing the source for a page. All we have to do is intercept that string before ASP.NET sends it to the browser and run our replacements. And it really doesn't take all that much code.

Since we want this mechanism to be available to all of the pages in an application, we'll create a new class named TokenReplacementPage that derives from System.Web.UI.Page. Any pages requiring token replacement functionality just need to derive from TokenReplacementPage instead of System.Web.UI.Page. Following is the code for the TokenReplacementPage class:

Code Listing 1 – TokenReplacementPage class

using System;
using System.IO;
using System.Text;
using System.Web.UI;

public abstract class TokenReplacementPage : Page
{
    protected override void Render(HtmlTextWriter writer)
    {
        //Create our own mechanism to store the page output
        StringBuilder pageSource = new StringBuilder();
        StringWriter sw = new StringWriter(pageSource);
        HtmlTextWriter htmlWriter = new HtmlTextWriter(sw);
        base.Render(htmlWriter);

        //Run replacements
        RunPageReplacements(pageSource);
        RunGlobalReplacements(pageSource);

        //Output replacements
        writer.Write(pageSource.ToString());
    }

    protected void RunGlobalReplacements(StringBuilder pageSource)
    {
        pageSource.Replace("[$SITECONTACT$]", "John Smith");
        pageSource.Replace("[$SITEEMAIL$]", "john.smith@somecompany.com");
        pageSource.Replace("[$CURRENTDATE$]",
            DateTime.Now.ToString("MM/dd/yyyy"));
    }

    protected virtual void RunPageReplacements(StringBuilder pageSource)
    {
    }

}

First, note that the TokenReplacementPage class is an abstract class that derives from System.Web.UI.Page. This means that it has all the standard Page functionality as well as token replacement features. All you have to do to confer token replacement functionality to a page is inherit from the TokenReplacementPage class instead of the Page class.

There are three methods inside the TokenReplacementPage class:

  • The Render method – to write the HTML page source to a StringBuilder
  • The RunGlobalReplacements method – to perform global token replacements on the source
  • RunPageReplacements – to allow for page-specific token replacements

Writing the Page Source to StringBuilder

The overridden Render method has a single HtmlTextWriter parameter named writer. An HtmlTextWriter allows you to write HTML to a stream. In this case, the underlying stream is sent to the browser, so the writer parameter is your conduit for outputting HTML to the people viewing your page. Normally, the Render method iterates through all of the controls on the page and passes the writer parameter to the Render method on each individual control. As each control executes its Render method, it writes the HTML for that section of the page to the browser.

Since we want to capture the entire page source before it gets to the browser, we need to do a little work to re-route the page source into a stream that we can access. To start, we create a "stream" that we can work with: a StringBuilder instance named pageSource. Next, we create a new StringWriter named sw and pass pageSource into its constructor. This initializes sw with pageSource as its underlying stream. Anything written to sw is output to pageSource. Next, we create a new HtmlTextWriter named htmlWriter and pass sw into its constructor. This initializes htmlWriter with sw as its underlying TextWriter. Thus, anything written to htmlWriter is written to sw, which is then written to pageSource. Finally, we pass htmlWriter to the Base.Render method and allow the page to render as it normally would. After Base.Render finishes, the entire page source is available for modification in pageSource.

After acquiring the page source, the overridden Render method passes pageSource to two methods that are responsible for actually making the substitutions: RunPageReplacements and RunGlobalReplacements. We will discuss these in more detail shortly.

Once the replacements have been made, the only thing left to do is send the updated content to the browser. We do that by using pageSource to write in the last line of code in the overridden Render method.

Next, let's take a look at making the actual replacements.

Global Replacements

You make global replacements, that is the replacements you want to make on every page that inherits from the TokenReplacementPage class, in the RunGlobalReplacements method. All you have to do to make a replacement is call the Replace method on the pageSource parameter and pass in the token and the replacement value:

pageSource.Replace("TOKEN", "REPLACEMENT VALUE");

The Replace method then searches through the string in pageSource and replaces any instances of the token with the replacement value. Remember to make your token something that is unlikely to normally appear in the page. For example, [$NAME$] is a much better choice than just NAME because it's very unlikely a normal sentence would contain a word with brackets and dollar signs around it. You don't want to accidentally mistake a normal word in a sentence for a token.

Page Specific Replacements

There may be times when you want to run page-specific replacements. Notice that the RunPageReplacements method in the TokenReplacementPage class is marked as virtual and contains no code. This allows you to override the RunPageReplacements method on the page in which you want to make page-specific token replacements. The replacements are made in the exact same fashion as described in the Global Replacements section, but they are only applied to that specific page:

Code Listing 2 – Overridden RunPageReplacements Example

public partial class PageSpecificReplacementsPage : TokenReplacementPage
{
    protected override void RunPageReplacements(
      System.Text.StringBuilder pageSource)
    {
        pageSource.Replace("[$PAGESPECIFICTOKEN$]",
            "This replacement only runs on this page!");
    }
}

Checking out the demo application

Download the demo application (from the Code Download link in the box to the right of the article title) and extract it to a location of your choosing on your hard drive. Start Visual Studio, and open the web site from wherever it is that you chose to save it. There are only four files (not including code-behinds) in the entire demo:

File Name

Purpose

App_Code\TokenReplacementPage.cs

Contains the TokenReplacementPage class that provides pages with token replacement functionality

Default.aspx

Demonstrates global token replacement functionality

PageSpecificReplacementsPage.aspx

Demonstrates global and page-specific token replacement functionality

Web.config

Website configuration

Take a look at the markup in the two ASP.NET pages and notice the various tokens that appear throughout. Also take a look at each page's code behind files because you will see a token set in code to demonstrate that you can put a token anywhere and, as long as it is output to the page source, it is replaced by the token replacement mechanism. The code-behind for PageSpecificReplacementsPage.aspx also contains the RunPageReplacement override from Code Listing 2.

When you run the demo application, you will see that the tokens are replaced with their respective values when the page appears in your browser. Make sure to observe the difference between the [$PAGESPECIFICTOKEN$] behavior between Default.aspx and PageSpecificReplacementsPage.aspx. Feel free to add new tokens, replacement values, and pages to the demo application to get a feel for how it all works.

Conclusion

When you need a token replacement mechanism, you should be well equipped with this solution. It gives you the ability to replace tokens regardless of whether they appear in ASP.NET controls, HTML controls, static HTML markup, code, or even a content management database. As long as you can get the token to render on the page, it can be replaced.

Damon Armstrong

Author profile:

Damon Armstrong is a Senior Engineering Team Lead with GimmalSoft in Dallas, Texas, and author of Pro ASP.NET 2.0 Website Programming. He specializes in Microsoft technologies with a focus on SharePoint and ASP.NET. When not staying up all night coding, he can be found playing disc golf, softball, working on something for Carrollton Young Life, or recovering from staying up all night coding.

Search for other articles by Damon Armstrong

Rate this article:   Avg rating: from a total of 58 votes.


Poor

OK

Good

Great

Must read
Have Your Say
Do you have an opinion on this article? Then add your comment below:
You must be logged in to post to this forum

Click here to log in.


Subject: Regex replacement is more flexible
Posted by: Anonymous (not signed in)
Posted on: Thursday, November 02, 2006 at 4:59 AM
Message: While the stringbuilder approach works fine for simple tokens, I have often found the need for more advanced parsing of the tokens. Using regular expressions allows you to create tokens that also include parameters which is a very powerful technique. For example you could use a token like [$Person:Name$] to insert the Name property of a person class. This technique works great where you have static or cached classes that contain globally accessible data. This is just one example of how you can extend the token replacement using regular expressions.

Subject: Regex Replacements - Next Article
Posted by: Damon (view profile)
Posted on: Thursday, November 02, 2006 at 2:04 PM
Message: I couldn't agree more. This is actually a primer for an article on Regular Expression based Token Replacement that will be published shortly.

Subject: Performance issues
Posted by: nuri (view profile)
Posted on: Tuesday, November 07, 2006 at 10:44 PM
Message: Your approach is managable from a software maintenance point of view, and would be friendly to developers.
When it comes to performance though, it would encounter more GC than necessary and some CPU overhead.

The main points are:
1) Your code attempts to find the token each time, even though the configured "static" text is rather static (and can be cached or read from file once)
2) Your code, even though it uses a string builder, ends up allocating memory repetedly that would be abandoned shortly after. This practice leaves large amounts of objects for GC to handle and ultimately degrades performance.

Please see
http://geekswithblogs.net/nuri/archive/2006/08/21/88586.aspx for demo code that attempts to improve on both these issues.

Regex is usefull in this context, but with caviates:
Regex should be compiled and cached. That is, keep a reference to a regex object, and in a static constructor set it to the "template" and compile so that the regex doesn't end up costing you again and again in parsing each page hit.

Secondly, Regex.Replace can be slower than string.replace on some short strings. (My code example mentioned above doesn't improve performance on short strings either vs. the "streightforward" approach btw).

A best of bread solution IMHO would:
1) Parse the template once (or infrequently, boot time etc)
2) Cache the template so IO and parsing is NOT encountered each page hit
3) Expose an interface to the coder that manages token names to mimic strong typing as much as possible. If a programmer uses a token that is not present in the template, a compile time or early runtime exception should occur. (My code raises a runtime if the runtime substitution token name doesn't appear in the template but does not alert if a token value is not submitted)

Sorry about the long comment, hope it makes sense and helps.

Nuri

Subject: Next Article - How Soon???
Posted by: SAinCA (view profile)
Posted on: Wednesday, November 08, 2006 at 6:10 PM
Message: Am tasked with glabal token replacement right now, so the follow-up on Regex based TR will likely save the day, not to mention hours of work going down _this_ article's path.

When, Damon, are you scheduled to produce the next article??? Do say, "very soon", as in, by Nov 30th :)

Why the rush? I'm a DB Developer, rusty in C++, never coded C#, HATE wasted effort when there really is a better way of doing something, and how we retrieve the base document and the token/value data are important design considerations for right now!

All haste, my friend! The presses await... and the consumers are hungry!

Thanks in advance.

Subject: Paul
Posted by: Anonymous (not signed in)
Posted on: Thursday, November 09, 2006 at 2:01 AM
Message: Great Article. I've been trying to do something similar in my content management system although I was thinking of having my tokens represent usercontrols. An example:

[$LatestNews.ascx(Section=2,From=1/1/2006,To=1/2/2006)$]

The idea is I would load the content from the db (which may include many tokens like above. And then for each token, instantiate the appropriate user control, set the properties and then add it into the control tree.

My problem was how to add this into the control tree and have it appear in the right order as if the tokens were in the middle of plain html from a db, none of the markup from the DB would be accessible from server controls.

Any ideas how this could be achieved. I was thinking maybe use literal controls to wrap the html that isn't tokenized in the db content

Subject: RE: Next Article - How Soon???
Posted by: Damon (view profile)
Posted on: Friday, November 10, 2006 at 11:13 PM
Message: I'm shooting to have it written by the end of November, and then there is normally about a week before it gets published to Simple-Talk.com

But, I already have the code for it done, just not the article that explains it. So go this page:

http://www.simple-talk.com/community/user/Profile.aspx?UserID=2140

And click on the "Send Email" link and I will be glad to send it to you "in the raw" so to speak.

Subject: Download code
Posted by: eliassal (view profile)
Posted on: Sunday, November 12, 2006 at 11:01 AM
Message: I can not find the download link. Below the stars at the top of the page are 4 icons. One of them for votes, the 2nd for comments, 3rd for send to, the 4th is for printing,no link for the source code.
Thanks

Subject: RE: Download Code
Posted by: Damon (view profile)
Posted on: Monday, November 13, 2006 at 9:42 AM
Message: I've emailed Tony (Simple-Talk Editor) and asked him to provide the link. In the meantime, you can use the email me link on this page:

http://www.simple-talk.com/community/user/Profile.aspx?UserID=2140

And I can send you the code via email. Sorry for the mishap.

Subject: re: Code Download
Posted by: Tony Davis (view profile)
Posted on: Monday, November 13, 2006 at 10:42 AM
Message: This was my mistake -- apologies for any inconvenience. I've now posted the code and you should ses a "Code Download" link in that box to the right of the article title.

Best,

Tony (Simple-Talk Ed)

Subject: Nice Article
Posted by: Anonymous (not signed in)
Posted on: Monday, November 20, 2006 at 5:56 AM
Message: Hey Friend ...

Nice article

Thanks

Somnath Mali
Ameriteck
http://www.ameriteck.biz

Subject: Nice Article
Posted by: benedict_kmu (view profile)
Posted on: Monday, November 20, 2006 at 11:52 PM
Message: Hey Damon,

Much helpful article.

Subject: Interesting Article
Posted by: Anonymous (not signed in)
Posted on: Tuesday, October 09, 2007 at 6:07 AM
Message: Damon

Just read through this article, and think it is great. I think I may implement something similar.


Subject: Great
Posted by: al_guiven (view profile)
Posted on: Tuesday, September 30, 2008 at 1:52 AM
Message: This article is great. Very informative

Subject: Alternate approach
Posted by: simpledev (view profile)
Posted on: Friday, October 24, 2008 at 4:12 AM
Message: It will change complete code. If you want to change tokens only in text/value property of control see here
http://urenjoy.blogspot.com/2008/10/set-tokens-for-dynamic-content.html

Subject: How do you change tokens in an email?
Posted by: hollyquinn (view profile)
Posted on: Thursday, June 25, 2009 at 4:25 PM
Message: This is a great article, and I'm having no problems replacing tokens on the page, but when I pass the string to a class where it is sent via an smtp message the tokens are not replaced. How do you use this inside of emails that you send out from your site? Is there a tutorial anywhere on this?

 

Top Rated

Acceptance Testing with FitNesse: Multiplicities and Comparisons
 FitNesse is one of the most popular tools for unit testing since it is designed with a Wiki-style... Read more...

Acceptance Testing with FitNesse: Symbols, Variables and Code-behind Styles
 Although FitNesse can be used as a generic automated testing tool for both applications and databases,... Read more...

Building Performance Metrics into ASP.NET MVC Applications
 When you're instrumenting an ASP.NET MVC or Web API application to monitor its performance while it is... Read more...

Acceptance Testing with FitNesse: Documentation and Infrastructure
 FitNesse is a popular general-purpose wiki-based framework for writing acceptance tests for software... Read more...

TortoiseSVN and Subversion Cookbook Part 11: Subversion and Oracle
 It is only recently that the tools have existed to make source-control easy for database developers.... Read more...

Most Viewed

A Complete URL Rewriting Solution for ASP.NET 2.0
 Ever wondered whether it's possible to create neater URLS, free of bulky Query String parameters?... Read more...

Visual Studio Setup - projects and custom actions
 This article describes the kinds of custom actions that can be used in your Visual Studio setup project. Read more...

.NET Application Architecture: the Data Access Layer
 Find out how to design a robust data access layer for your .NET applications. Read more...

Calling Cross Domain Web Services in AJAX
 The latest craze for mashups involves making cross-domain calls to Web Services from APIs made publicly... Read more...

Web Parts in ASP.NET 2.0
 Most Web Parts implementations allow users to create a single portal page where they can personalize... Read more...

Why Join

Over 400,000 Microsoft professionals subscribe to the Simple-Talk technical journal. Join today, it's fast, simple, free and secure.