We recently published a methodology specific to evasions testing at NSS Labs. Evasions testing is a category of the Security Effectiveness testing that is included in most of our test methodologies. This new evasions test methodology provides a more detailed description of how NSS performs evasions testing and highlights key aspects of our private and public (or “group”) tests.
Private testing is an opportunity for vendors to have their products tested by NSS in order to identify and remediate any defects in GA or non-GA code builds, and/or new or existing hardware, with the knowledge that these results will not be made public. Private tests also provide vendors and enterprises with visibility into the robustness of products and policies prior to deployment. These engagements are paid for and protected under NDA. Private tests competently exercise all aspects of a product against NSS’ current test methodology and reveal areas that require improvement. Public (group) tests are not paid for, are executed against current methodologies—and fully exercise all aspects of a product under that methodology; are only performed with GA code on commercially-available hardware, and the results are made public.
To ensure that any issues identified in previous testing have been adequately addressed, and to confirm that products are indeed providing the level of protection claimed (and not just becoming more proficient at taking the same test), we modify our testing between private and public test cycles for any given test methodology.