Improve xccdf_results_mapper when converting XCCDF Results to HDF Res…

…ults (#4255)

* Improve xccdf_results_mapper when converting XCCDF->HDF

* Support XCCDF generated using ComplianceAsCode:
  https://github.com/ComplianceAsCode/content/
* Support nested XCCDF groups
* Set `code_desc` to an appropriate value (not empty string)
* Refactor to remove global variables

Bug: mitre/saf#1163
Fixes: #4194

* Minor changes requested by code review

* Use the "RegExp.exec()" method instead

Address sonar finding typescript:S6594

* Set impact to 0 for 'notapplicable' and 'informational' results

* Don't handle every array item within each array item

In handleArray, the array v is looped over. For each item, the entire
array is looped over again, resulting in an array v of length n
resulting in an output array of length n^2, making handleArray's
complexity O(n^2).

However, that n^2 looping is unnecessary. Removing it brings the
complexity of handleArray down to O(n), drastically reducing execution
time.

* "version" should use "version.text" (not just "version")

* For version, prefer version over id

* For version, remove unnecessary comment

* Remove unnecessary String conversion

* Add tsdoc to getRulesInGroup

* removed 'id' as a potential path for 'version'.  the complianceascode sample doesn't contain that attribute at all -> we should not have a tag for it.  also updated samples but these samples updates also include changes from the baseconverter updates so warrant inspection esp for the complianceascode sample output

Signed-off-by: Amndeep Singh Mann <[email protected]>

* linting

Signed-off-by: Amndeep Singh Mann <[email protected]>

* Use triple equals for string comparson

* Various fixes

* Correct "refs" to comply with schema

* Only include description if it has a label

* make the nist family part of the regexes only match against valid nist control families

Signed-off-by: Amndeep Singh Mann <[email protected]>

* Use `as unknown as ControlDescription` instead of `as any`

* Run lint on src/nist.ts

Fixes linting issues introduced in 587282d

* Regenerate samples

Updates samples after 587282d

* Use concise character class syntax '\d' instead of '[0-9]'.

Addresses sonar finding typescript:S6353

* the treemap expects a canonized form of the nist controls that are not zero-padded.  also changed default behaviors so there's no need to supply a parameter to the canonize function at all.

Signed-off-by: Amndeep Singh Mann <[email protected]>

* get rid of dupe nist tags - even if there were dupes in the original file, it makes no sense to have that replicated here anymore really - there are some particularly egregious cases too where a control is replicated at least 3 times in the array

Signed-off-by: Amndeep Singh Mann <[email protected]>

* could simplify the default_partial_config implementation and also ran the linter

Signed-off-by: Amndeep Singh Mann <[email protected]>

* sonarqube

Signed-off-by: Amndeep Singh Mann <[email protected]>

---------

Signed-off-by: Amndeep Singh Mann <[email protected]>
Co-authored-by: Aaron Lippold <[email protected]>
Co-authored-by: Amndeep Singh Mann <[email protected]>

libs/hdf-converters/test/mappers/forward/xccdf_mapper.spec.ts

@@ -54,6 +54,30 @@ describe('xccdf_mapper', () => {
     });
   });
   describe('OpenSCAP', () => {
+    it('ubuntu1804', () => {
+      const mapper = new XCCDFResultsMapper(
+        fs.readFileSync(
+          'sample_jsons/xccdf_results_mapper/sample_input_report/xccdf-results-openscap-ComplianceAsCode-ubuntu1804.xml',
+          {encoding: 'utf-8'}
+        )
+      );
+
+      // fs.writeFileSync(
+      //   'sample_jsons/xccdf_results_mapper/xccdf-openscap-ComplianceAsCode-ubuntu1804-hdf.json',
+      //   JSON.stringify(mapper.toHdf(), null, 2)
+      // );
+
+      expect(omitVersions(mapper.toHdf())).toEqual(
+        omitVersions(
+          JSON.parse(
+            fs.readFileSync(
+              'sample_jsons/xccdf_results_mapper/xccdf-openscap-ComplianceAsCode-ubuntu1804-hdf.json',
+              {encoding: 'utf-8'}
+            )
+          )
+        )
+      );
+    });
     it('rhel7', () => {
       const mapper = new XCCDFResultsMapper(
         fs.readFileSync(

libs/inspecjs/src/nist.ts

@@ -2,33 +2,38 @@ import {ControlStatus} from './compat_wrappers';
 import {ALL_NIST_CONTROL_NUMBERS, ALL_NIST_FAMILIES} from './raw_nist';
 
 // Regexes.
-const NIST_FAMILY_RE = /^[A-Z]{2}$/;
+// Matches against only currently existing NIST Control families instead of allowing any two adjacent uppercase letters
+const NIST_FAMILY_RE =
+  /^(?:A[CPRTU]|C[AMP]|D[IM]|I[APR]|M[AP]|P[ELMS]|RA|S[ACEI]|TR|U[LM])$/;
 // Limit length of children to avoid potential DoS on malicious NIST Control strings
-const NIST_CONTROL_RE = /^([A-Z]{2})-([0-9]+)(.{0,60})$/;
+const NIST_CONTROL_RE =
+  /^(A[CPRTU]|C[AMP]|D[IM]|I[APR]|M[AP]|P[ELMS]|RA|S[ACEI]|TR|U[LM])-(\d+)(.{0,60})$/;
 const SPEC_SPLITTER = /[\s\(\)\.]+/; // Includes all whitespace, periods, and parenthesis
 const REV_RE = /^rev[\s_.]+(\d+)$/i; // Matches Rev_5 etc
 type ParseNist = NistControl | NistRevision | null;
 
 export interface CanonizationConfig {
-  max_specifiers: number;
-
-  // All are assumed false
+  max_specifiers?: number; // default 5: $ rg '<number>' SP_800-53_v5_1_XML.xml | awk -F'[^ ]' '{print length($1)}' | sort -nr | head -1 | xargs -I{} expr \( {} - 6 \) / 3 # this equals 5 as of rev5
   pad_zeros?: boolean; // default false
   allow_letters?: boolean; // default true
   add_spaces?: boolean; // default true
   add_parens?: boolean; // default true
   add_periods?: boolean; // default true
 }
 
-function default_partial_config(c: CanonizationConfig): CanonizationConfig {
-  return {
-    pad_zeros: false,
-    allow_letters: true,
-    add_spaces: true,
-    add_parens: true,
-    add_periods: true,
-    ...c
-  };
+export const DEFAULT_CANONIZATION_CONFIG = {
+  max_specifiers: 5,
+  pad_zeros: false,
+  allow_letters: true,
+  add_spaces: true,
+  add_parens: true,
+  add_periods: true
+};
+
+function default_partial_config(
+  c?: CanonizationConfig
+): Required<CanonizationConfig> {
+  return {...DEFAULT_CANONIZATION_CONFIG, ...c};
 }
 
 /** Represents a single nist control, or group of controls if the sub specs are vague enoug. */
@@ -123,8 +128,8 @@ export class NistControl {
    * This is, unfortunately, slightly expensive.
    * Avoid repeating this if possible.
    */
-  canonize(config: CanonizationConfig): string {
-    config = default_partial_config(config);
+  canonize(c?: CanonizationConfig): string {
+    const config = default_partial_config(c);
     const ss = this.subSpecifiers;
 
     // Build our string. Start with family