25 June 2013

How To Document Your PowerShell Library

PowerShell provides comment-based help for functions and scripts with Get-Help, but when you want to document modules or generate a complete indexed API in HTML format, just as you can with Sandcastle for .NET or javadoc for Java, then stronger magic is required. Michael Sorens shows you how it is done, with source code.

Contents

I am passionate about documentation. When you write library code-code for you or others to consume in other projects-it is almost useless without proper documentation. Most modern languages provide some level of support to assist in creating your documentation, where you instrument your code with documentation comments (doc-comments for short) that may then be automatically extracted and formatted to generate some documentation. But documenting individual methods or classes or files is not the same as documenting an API. Java has javadoc and C# has Sandcastle, both of which generate a complete, cross-linked API documentation set. On the other hand, languages like Perl with perldoc and PowerShell with Get-Help, generate only isolated module documentation, which I found rather unsatisfactory. So over the years I created an API-level documentation generator for Perl, later for T-SQL, and most recently for PowerShell. (Note that the Perl version Pod2HtmlTree is Perl-specific while the T-SQL one uses my generic XML conversion tool XmlTransform configured to handle SQL documentation, described in Add Custom XML Documentation Capability To Your SQL Code.) The PowerShell generator (written in PowerShell, of course!) is a function in my DocTreeGenerator module called Convert-HelpToHtmlTree. (You can find my entire API bookshelf here and download libraries here.)

This article discusses how to generate a complete API for a PowerShell library. Once you have instrumented your modules with doc-comments to satisfy Get-Help, you need surprisingly little extra work to supply to Convert-HelpToHtmlTree. But let’s start with the assumption, hardly realistic, I’m sure, that you do not have any doc-comments in your code and see what you can already do.

Test Driving the Generator

To help you visualize what Convert-HelpToHtmlTree does, consider this simple example. Assume you have just one module, MathStuff.psm1 containing two functions:

Notice that there is as yet no documentation-comments in the code to document these functions. Yet PowerShell’s standard help cmdlet, Get-Help, can still give you some information:

Similarly, Convert-HelpToHtmlTree can give you all the same basic information about the individual method-but it also builds an entire web framework around the namespace and module that contains it. Here is a snapshot of the input directory tree. By PowerShell convention, store your user modules under WindowsPowerShell\Modules in your user directory. Under that path follows the namespace for this example, DemoModules. There are two module directories, MathStuff and StringUtilities; the former defines the two functions shown earlier, while the latter is empty.

Now execute Convert-HelpToHtmlTree with minimal arguments: the namespace to document and where to put it. Convert-HelpToHtmlTree needs more infrastructure to give you more useful output; it lets you know what you need to provide and where it goes:

Here is the generated output tree:

As is typical for a website, index.html is the entry point, displaying the namespace contents: a list of all contained modules with a brief description and a link to each module’s documentation (see Figure 1). In this case there is just one module, MathStuff, because StringUtilities mentioned above was empty.

1826-initial_web_home-69ae35e6-0a1b-48df

Figure 1 Sample namespaces page: itemizes the modules for each namespace you have specified to document. (This is your top-level, home, or entry page.)

If you traverse the link to MathStuff, you get the module contents: a list of all contained functions, and filters with a brief description and a link to each one’s documentation (see Figure 2). Notice in each of these figures, by the way, that you have the same warning messages inline as you did at the command prompt; here you get to see them in context. Figure 2 shows one of the few instances where Convert-HelpToHtmlTree fills in a slot where information is not available instead of giving a warning-in the table of methods it lists the syntax of a method in the absence of a textual description.

1826-initial_web_group-ceca7164-611e-4a0

Figure 2 Sample module page: itemizes the functions, filters, and aliases for the current module.

If you traverse the link to an individual function, you get the spruced-up output of Get-Help for that function(see Figure 3). This is exactly the same output that you get from Get-Help, but with these enhancements:

  • It adds section headings to each of the main sections.
  • It outputs most text in a proportional font.
  • It outputs portions of text you designate (via leading white space on a line) in a fixed-width font.
  • It preserves your line breaks, so if you want text to flow and wrap automatically at the window edge you must put it all in a single paragraph in your source file.
  • It highlights the initial code sample in each example. (Note that, just like Get-Help, ONLY the first line immediately following the .EXAMPLE directive is treated as the code sample, so resist the urge to put line breaks in your sample!)
  • Items designated as links are turned into active hyperlinks; these may be in any of seven different formats explained later.

1826-initial_web_method-dd51125c-6d5e-47

Figure 3 Sample function page: provides help text for the current function or filter. This is a stylized version of the output of Get-Help.

Finally, in the root folder of the output tree is contents.html, a table of contents, or index, if you will of everything (Figure 4).

1826-initial_web_contents-46408bcf-0562-

Figure 4 Sample master index/contents page: an index of all documented objects; this page is accessible via a hyperlink on the top-level namespaces page.

Controlling Output with the Page Template

Figures 1 through 4 show the four page-types that Convert-HelpToHtmlTree can generate. All four of these are derived from a single HTML template file. Take a look at the default template (psdoc_template.html) and you will find it sprinkled with place-holders for different properties. In the previous figures you will see some remnants where some place-holders have not been supplied, e.g. the empty copyright and revision dates at the bottom of each page.

Here is the content of the <body> element of the template (the <head> element has been omitted for brevity). I have distinguished the global properties, module properties, and conditional properties with different colors for clarity. The following sections describe each of these three property types in detail.

Global Properties

Global properties are distinguished by having a single word in brackets for the place holder.

Place Holder Source Applies to PageType
{title} current namespace + DocTitle parameter master, contents
{subtitle} current namespace + DocTitle parameter module, function
{breadcrumbs} generated relative paths module, function
{preamble} manifest.Description property + module_overview.html <body> contents module
{body} contents of Get-Help function
LIST(function name + Get-Help.Synopsis property) module
LIST(Namespace_overview.html <body> contents

   + LIST(module name + manifest.Description property))
master
generated index contents
{postscript}   -not currently used-
{copyright} Copyright parameter -ALL-
{revdate} RevisionDate parameter -ALL-

Module Properties

Module-specific properties are distinguished by the form {module.property} where property may be any of the standard properties of a module. You can see what these are with this command:

As an example, Figure 2 shows that the template uses the module.guid and module.version properties used.

Conditional Properties

Finally, you will also see conditional properties in the template of the form:

…where pagetype may be any of the four types of pages created:

  • the single master page (pagetype=master),
  • the single master index/contents page (pagetype=contents),
  • the module index pages, one per module (pagetype=module), and
  • the function pages, one per exported function (pagetype=function).

The content of these conditional sections (which may be any valid HTML) is included only on the pages of the corresponding type, while the other conditional sections are suppressed. Note that the module-specific place-holders discussed earlier (e.g. {module.property}) may be used in both module pages and function pages, but not in the master or contents page.

Specifying Links

Unlike the MSDN pages for the standard PowerShell library, output generated here creates live links in your references (.LINK) documentation section. There are seven input variations permitted:

Item MSDN-defined (built-in) cmdlet
Sample Input Get-ChildItem
Sample Output <a href=’http://technet.microsoft.com/en-us/library/dd347686.aspx’>Get-ChildItem</a>
Item MSDN-defined (built-in) topic
Sample Input about_Aliases
Sample Output <a href=’http://technet.microsoft.com/en-us/library/dd347645.aspx’>about_Aliases</a>
Item Custom function in the same module
Sample Input New-CustomFunction
Sample Output <a href=’New-CustomFunction.html’>New-CustomFunction</a>
Item Custom function in a different local module
Sample Input New-FunctionInOtherModule
Sample Output <a href=’../../DemoModules/MathStuff/New-FunctionInOtherModule.html’>New-FunctionInOtherModule</a>
Item Plain text (for cases where you have no web resource to link to)
Sample Input Butterflies and Dinosaurs–the missing link?
Sample Output Butterflies and Dinosaurs–the missing link?
Item Explicit link with a label
Sample Input [other important stuff] (http://foobar.com)
Sample Output <a href=’http://foobar.com’>other important stuff</a>
Item Explicit link without a label
Sample Input http://alpha/beta/
Sample Output <a href=’http://alpha/beta/’>http://alpha/beta/</a></

The MSDN references are automatically constructed by referencing two fixed MSDN reference pages, one for cmdlets and one for topics. If those fixed references ever change URLs, that will break the generator; you will need to update those URLs in the internal Get-CmdletDocLinks function to mend it. The input format for these was designed keeping in mind that while Convert-HelpToHtmlTree converts these to links, Get-Help does not-it will display the original, raw text.

Other Structural Considerations

Besides the standard documentation comments that Get-Help would use to display your modules, Convert-HelpToHtmlTree needs some additional doc-comments to generate a cohesive API for you.

  1. Each module (x.psm1) must have an associated manifest (x.psd1) in the same directory and the manifest must include a Description property.
  2. Each module must have an associated overview (module_overview.html) in the same directory. This is a standard HTML file. The contents of the <body> element are extracted verbatim as the introductory text of the index.html page for each module.
  3. Each namespace must also include an associated overview (namespace_overview.html). This is a standard HTML file. The contents of the <body> element are extracted verbatim as the introductory text of each namespace in the master index.html page.

By reviewing the error messages in the very beginning of the article as well as the table of global properties early on, you should now see how these pieces fit together.

Note that I use the term namespace here informally because (as of v3) PowerShell does not currently have the notion of namespaces. Convert-HelpToHtmlTree, however, requires you to structure your modules grouped in namespaces. Thus, if you have a module MyStuff.psm1, normal PowerShell conventions require you to store this in a path like this:

…\WindowsPowerShell\Modules\MyStuff\MyStuff.psm1

…but Convert-HelpToHtmlTree requires you to include one more level for namespace, so the module must be stored in a path like this:

…\WindowsPowerShell\Modules\MyNamespace\MyStuff\MyStuff.psm1

This allows you to organize your modules into more than one logical group if desired. In my own PowerShell library, for example, I have FileTools, SqlTools, and SvnTools modules (among others) all under the CleanCode namespace. But you may, however, include multiple namespaces. Here’s a sample input tree illustrating this:

The output structure mirrors the input structure; the above input might generate the output tree shown below. There is a single master index page documenting all namespaces.

Other Stylistic Considerations

You have seen how to customize the generated output with the template file and how to format items in your .LINKS section. There are just a few more considerations to be mindful of as you write your documentation comments for your modules, touched on earlier:

  • Body text, by default, is output with a proportional font (your standard browser font) and flows/wraps automatically at the window edge based on your paragraph boundaries-that is, where you have hard (explicit) carriage returns.
  • By inference, if you want to output a list of items, you do not need to do anything special. Just type in the list with carriage returns and the output will mirror the input. It will still be in the standard browser font.
  • If you want to include a code sample in a fixed-width font, or even if you just want to output a list of items in a fixed-width font, simply have leading whitespace on each line.
  • The first line-and only the first line-following an .EXAMPLE section is special to Get-Help: it is considered your code sample. Thus, you really want to keep your code sample all on one line, no matter how long and ugly it gets. Convert-HelpToHtmlTree also considers the same first line a code sample and highlights it. So resist the urge to put line breaks in your sample!

Now Just Add Water…

With the preliminaries out of the way, you are ready to put the generator to work. There are just a few parameters to specify on the command line. Here is essentially the script I use to generate the API for my own CleanCode PowerShell library:

This short script uses a custom template file located in the same directory as the above script. (If you omit specifying a template it uses the default template supplied. That template includes the CSS rules embedded in the template just to keep it all to a single file; in practice you should generally move the CSS rules to a separate file.)

Figure 5 shows you the API home page for my PowerShell libraries.

1826-cleancode_web_home-5cafa97c-4294-45

Figure 5 Actual API home page for my CleanCode library of PowerShell functions.

And Figure 6 shows a portion of its associated master index. From either of those you can browse through the remainder of the documentation set.

1826-cleancode_web_contents-2e01d05e-3f5

Figure 6 Portion of the index (or table of contents) of the CleanCode library, indexing namespaces, modules, functions, and filters.

1826-side%20by%20side%201-d28aaeb5-2091-

Figure 7 Side by side comparison of the stylized web page output of Convert-HelpToHtmlTree (left) and Get-Help (right). Note the section headers standout for easy scanning. The description shows both normal, wrapped text as well as an example of preformatted text.

Conclusion

Some IDEs provide great support for adding doc-comments. Visual Studio, for example, with the GhostDoc add-on almost writes the doc-comments for you. Alas, PowerShell does not yet have such a “ghost writer” available. To do it yourself, start with about_Comment_Based_Help (which you can also access from the PowerShell prompt by feeding that to Get-Help!). Scroll down to the Syntax for Comment-Based Help in Functions section. Note that the page also talks about adding help for the script itself; that applies only to script files (ps1 files); it does not apply to modules (psm1 files). What you will see here is that you must add a special comment section that looks like this for each function (those that may appear more than once are shown repeated here):

After each keyword directive, you can have as much freeform text as you need. You will find all the keywords described in the subsequent section of about_Comment_Based_Help entitled Comment-Based Help Keywords.

Now go forth and document!

Update – April 2016

Since this article, I’ve created a wallchart putting both XmlDoc2Cmdlet and DocTreeGenerator in context, showing you how to do a complete documentation solution for your PowerShell work in both C# and PowerShell. Click here for more details.
2407-1-5728ddc3-433a-4b82-a6dc-84f5f450b

Keep up to date with Simple-Talk

For more articles like this delivered fortnightly, sign up to the Simple-Talk newsletter

This post has been viewed 27446 times – thanks for reading.

Tags: , , , , ,

  • Rate
    [Total: 13    Average: 4.8/5]
  • Share

Michael Sorens

Michael Sorens is passionate about software to be more productive, evidenced by his open source libraries in several languages (see his API bookshelf) as well as SqlDiffFramework (a DB comparison tool for heterogeneous systems including SQL Server, Oracle, and MySql). With degrees in computer science and engineering he has worked the gamut of companies from Fortune 500 firms to Silicon Valley startups over the last 25 years or so. Current passions include PowerShell, .NET, SQL, and XML technologies (see his full brand page). Spreading the seeds of good design wherever possible, he enjoys sharing knowledge via writing (see his full list of articles), teaching, and StackOverflow. Like what you have read? Connect with Michael on LinkedIn and Google +

View all articles by Michael Sorens

  • Toolsmith

    Thank you!
    I’ve been thinking for a while now that my library of PowerShell scripts is getting out of hand (mostly since I have to re-read each script before I run it to remember what the heck it was for).

    This article is definately a kick-in-the-pants reminder that I need to be more strict with myself on my "one time" scripts!

  • drohm

    Templates
    Are there any HTML templates available? Yes, I’m lazy 🙂

    Thanks for this great tool!

    Doug

  • msorens

    Re: Templates
    Doug: There is a basic template included with the code, as explained in the article. Beyond that, I have only one other template that I use to generate the public API for my PowerShell library. I would be happy to send that to you–send me a note on LinkedIn with your email address if you like.

  • ChrisLynch

    Pretty cool
    This is one of the better tools I have found on how to create HTML online documentation for cmdlets. Can you provide any guidance on what would need to change in your DocTreeGenerator module to generate Codeplex-compliant HTML?

  • msorens

    Re: Codeplex-compliant HTML
    @ChrisLynch: Neat question! I infer that you want to document a PowerShell library hosted on CodePlex and post that generated documentation on your CodePlex site itself. As you are no doubt aware, CodePlex uses a fairly standard wiki markup language. So the question becomes how to output wiki markup instead of outputting HTML.

    You will need to address two pieces:
    (1) Modify the template file psdoc_template.html so the boilerplate uses wiki notation instead of HTML; and
    (2) Modify the generator itself to emit markup instead of HTML. With clever forethought :-), I have encapsulated all the HTML-specific output generation into the last section of the code (about the last 100 lines, and most of them are one-line functions). Here is an example: consider the ‘Get-HtmlLink’ function, which is defined as just:

    "<a href='{0}’>{1}</a>" -f $href, $value

    To emit wiki markup, just change the body of that function to this:

    [url:$value|$href]

    I am not sure if everything will map quite that easily, though! Let me know how it goes, either here or connect with me via LinkedIn.

  • Jason Thurston

    Module Path Requirement
    I have a workspace project with it’s own "Modules" folder, I have a <project>-Profile.ps1 file that I run to add this Module folder to $env:psmodulepath but I cannot get Convert-HelpToHtmlTree to generate docs against those modules. Is there a way to make it work this way or do I always have to temporarily copy my project’s modules to C:Users/<user>/Documents/WindowsPowerShell/Modules/ any time I want to generate the documentation?

  • msorens

    Re: Module Path Requirement
    @Jason: Short answer: yes. Currently modules must be under the standard user path you have indicated. (As documented in the Convert-HelpToHtmlTree help.)

    However, I realize that may be an issue for some folks, so I have just entered this as an issue (see https://github.com/msorens/DocTreeGenerator/issues/4). If interest warrants, I also think it would be a useful update.

  • Jason Thurston

    Module Path Requirement
    I have a workspace project with it’s own "Modules" folder, I have a <project>-Profile.ps1 file that I run to add this Module folder to $env:psmodulepath but I cannot get Convert-HelpToHtmlTree to generate docs against those modules. Is there a way to make it work this way or do I always have to temporarily copy my project’s modules to C:Users/<user>/Documents/WindowsPowerShell/Modules/ any time I want to generate the documentation?

  • Jason Thurston

    *_overview.html file format
    Discovered that the *overview_html files have to be valid xml files with a <body> tag, not just any plain html file. Also, there must an xml tag inside the <body> tag. You can’t just have plain text in the body.

    It seems that the main header comment in a module doesn’t show up in the documentation so I use a wrapper script that calls convert-helpthtmltree that parses out the main header comment in the modules, html escapes it, wraps it in a <pre> tag and puts all of that into the <body> tag of the module_overview.html file for each of my modules. This wrapper also copies the modues from my project home to my user profile module home.

  • drohm

    Re: Module Path Requirement
    Jason,
    Here is a simple script I use when generating the help for our modules. It does the following:

    1. Pulls the latest source from git.
    2. Copies DocTreeGenerator specific files to $PROFILEModulesDocTreeGenerator
    3. Copies the modules source to $PROFILEModulesRELENG (for namespacing).
    4. Calls Convert-HelpToHtmlTree on the modules using the -TargetDir parameter to output the help to a local IIS folder for viewing online.
    5. Cleanup – removes all files copied to $PROFILE folder.

    View it here – https://pastebin.com/5JqbDBpW

    This is set as a scheduled task that runs nightly. Hopefully it helps.

    Doug

  • msorens

    Re: *_overview.html file format
    On your first point (plain text within the <body> element is invalid), that was designed to be consistent with valid HTML structure. However, I am updating the cmdlet help to explicitly state that requirement.

    On your second point, I would be interested to see an example of what you mean by "main header comment". It might be something to address in a future update as well.

    Thanks for your continuing feedback!

  • msorens

    Re: Module Path Requirement
    @Doug: Thanks for sharing your wrapper script. I am sure other readers will appreciate it as well !

  • Jason Thurston

    *_overview.html file format
    Discovered that the *overview_html files have to be valid xml files with a <body> tag, not just any plain html file. Also, there must an xml tag inside the <body> tag. You can’t just have plain text in the body.

    It seems that the main header comment in a module doesn’t show up in the documentation so I use a wrapper script that calls convert-helpthtmltree that parses out the main header comment in the modules, html escapes it, wraps it in a <pre> tag and puts all of that into the <body> tag of the module_overview.html file for each of my modules. This wrapper also copies the modues from my project home to my user profile module home.