Suggestion for output...

Dec 31, 2009 at 11:12 PM


Just a couple of quick suggestions for speeding up the whole unit-test flow be doing some tweaks to by-pass the formatted output.  I have a series of tests that has been taking an avg of 6.5-7 seconds to run and decided that was too long.  So, I added a switch -RawOutput to PSUnit.Run.ps1 and then wrapped the formatting 'Switch' statement to be by-passed.  Not sure if you'd be interested in allowing for straight -ole output but it sure helped increased the speed and gave me much wanted versatility.

Also, while doing that a per-test time was added to the test object, and the formatted output.  Now when using the raw format, the user can filter/match/sort etc till the heart is content.  As an example:


# run-tests is just a lil function to do the formatting and wrap the whole thing up in a measure command block to compare to the normal output:
# function run-tests {Measure-Command {$TRs=(C:\scripts\FuncGuardTestStarter.ps1); %{$TRs | group Result | sort Name -Descending | foreach {$_.Group|Sort Time| Format-Table -AutoSize}; $TRs | measure-object Time -ave -max -sum}|out-host}}
PS C:\Windows\system32> run-tests

Test                                                          Result Reason    Time
----                                                          ------ ------    ----
Test.testNameParam_Valid__FuncName_NamePart1_1NamePart        PASS           7.0004
Test.testNameParam_Valid__FuncName_NamePart_NamePart_1        PASS           7.0004
Test.testNameParam_Valid__FuncName_NamePart_NamePart          PASS           8.0004
Test.testPrefixParam_ValidWithDigit_PRE01                     PASS           8.0004
Test.testPrefixParam_ValidWithDigit_PRE01FIX02                PASS           8.0004
Test.testNameParam_Valid__FuncName                            PASS           9.0005
Test.testNameParam_Valid__FuncName_Part                       PASS           9.0005
Test.testNameParam_Valid__FuncName_1NamePart_NamePart         PASS           9.0005
Test.testPrefixParam_ValidLower_Prefix                        PASS           9.0005
Test.testNameParam_Valid__FuncName_NamePart_NamePart1         PASS           9.0005
Test.testNameParam_Valid__FuncName_NamePart1_NamePart         PASS           9.0005
Test.testNameParam_Valid__FuncName_NamePart                   PASS           9.0006
Test.testNameParam_Valid__FuncName_NamePart_1_NamePart        PASS          10.0006
Test.testNameParam_Valid__FuncName_NamePart_1NamePart         PASS          10.0006
Test.testPathParam_ValidPath                                  PASS          10.0006
Test.testPrefixParam_Valid_PREFIX                             PASS          11.0006
Test.testPrefixParam_InValidLeadingDigit__1PREFIX             PASS          11.0006
Test.testNameParam_InValidUpper__Func_NAME                    PASS          11.0006
Test.testPrefixParam_InValidToShort__PR                       PASS          11.0007
Test.testPrefixParam_InValidChars__PRE_FIX                    PASS          12.0007
Test.testNameParam_InValidPunctation__Func_NamePercent        PASS          12.0007
Test.testPrefixParam_InValidNotEnoughLettersBeforeDigit__PR01 PASS          13.0007
Test.testPrefixParam_InValidToLong__ALONGPREFIX               PASS          13.0008
Test.testNameParam_InValidPartialUpper__FUnc_Name             PASS          13.0008
Test.testNameParam_InValidUpper__FUNCNAME                     PASS          13.0008
Test.testNameParam_InValidLower__funcname                     PASS          13.0008
Test.testNameParam_InValidLeading__1Func_Name                 PASS          14.0008
Test.testNameParam_InValidPartialUpper2__Func_NAme            PASS          14.0008
Test.testPathParam_InvalidPath                                PASS          15.0009
Test.testNameParam_InValidSpacing__Func__NAME                 PASS          16.0009
Test.getFuncGuardCoreFiles_ValidPathWithoutRequiredFiles      PASS           16.001
Test.testNameParam_InValidLower__Func_name                    PASS           18.001
Test.testNameParam_InValidPartialLower__fuNc_Name             PASS          18.0011
Test.testNameParam_InValidSpacing__Func_NAME_                 PASS          19.0011
Test.getFuncGuardTemplateFiles_ValidPathZeroFiles             PASS          50.0028
Test.getFuncGuardTemplateFiles_Valid                          PASS          58.0033
Test.getFuncGuardSourceFiles_Optional                         PASS          64.0037
Test.getFuncGuardCoreFiles_ValidPathWithRequiredFiles         PASS          65.0037
Test.Get-FuncGuard                                            PASS           69.004
Test.readFuncGuardHeader_InvalidFileFormat                    PASS          71.0041
Test.getFuncGuardSourceFiles_AllWithExplicitTemplatePath      PASS          75.0043
Test.getFuncGuardSourceFiles_AllWithDefaultTemplatePath       PASS          75.0043
Test.getFuncGuardCoreFiles_ValidPathWithoutSomeRequiredFiles  PASS          77.0044
Test.getFuncGuardSourceFiles_Required                         PASS          89.0051

Test                                                   Result Reason                                 Time
----                                                   ------ ------                                 ----
Test.testNameParam_Valid__FuncName1_NamePart_NamePart  FAIL   Invalid function guard name format. 12.0007
Test.testNameParam_Valid__FuncName1_1NamePart_NamePart FAIL   Invalid function guard name format. 13.0007

Count    : 46
Average  : 24.175302173913
Sum      : 1112.0639
Maximum  : 89.0051
Minimum  :
Property : Time

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 4
Milliseconds      : 23
Ticks             : 40235558
TotalDays         : 4.65689328703704E-05
TotalHours        : 0.00111765438888889
TotalMinutes      : 0.0670592633333333
TotalSeconds      : 4.0235558
TotalMilliseconds : 4023.5558




Another thing I did was eliminate the if statement in the PSUnit.Support.ps1 Encode-Html which also helped speed things up a tad.  In order to have both 32 and 64 bit work properly (I hope) it was changed to:



function Encode-Html([string] $StringToEncode)
	$AssemblyTarget=("System.Web, Version=, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a")
        [Void] [Reflection.Assembly]::Load($AssemblyTarget)
        Write-Debug "Assembly not found: [$AssemblyTarget]"
    $HtmlEncodedErrorRecordString = [System.Web.HttpUtility]::HtmlEncode($StringToEncode)
    return $HtmlEncodedErrorRecordString


Which isn't something I'd recommend to use for the catch block since if neither are available the exception should be re-thrown.  Right?


Hopefully a formatted output by-pass will be available so we can work with the objects in the pipe as we see fit.


Thanks again for the nice package!  It is really proving worth-while.


Jan 6, 2010 at 6:24 AM

Thanks for the feedback.

I will add the suggested improvements to the scripts.



Jan 14, 2010 at 5:38 AM

I am going to replace the code that loads of System.Web.dll with a call of the Add-Type cmdlet. This call will be done in once in the script scope and not part of the function. This should speed things up.

Jan 14, 2010 at 5:45 AM

And now I get it. What a great idea to output pipe out the test results. I think I will refactor the run script to output the test result objects and then create a new function / script that renders the test results as report.


Jan 19, 2010 at 4:55 AM

I've absolutely been getting more out of having the output objects rather than a custom formatted view.  Using filtering, grouping, summary views, etc...

While I haven't quite gone to the exent of adding a format.ps1xml or a types.ps1xml yet I think both are a good idea.


Looking forward to the new version...