Compliance example

Since Nodelab is based on the Edgescript® programming language, Nodelab can handle fairly complex and complicated tasks.

For instance, compliance standards can be massive. The PCI DSS compliance standard contains more than 200 requirements, each with a number of controls, looking into logs and snapshot files. The status for many of these controls must be evaluated for every configurable item in the environment.
This means that tens of thousands of status points must be calculated every time you perform an environment review. You want to do this continuously, not once per year, to keep track of improvements or other changes.
Since each control definition is unique, it must be dynamically implemented. A configuration definition that includes every control difference would be too complicated and hard to maintain.

A sample of PCI DSS definitions can be found here.

Nodelab can automate these tasks, using parameterized Edgescript® calls to a number of methods, and present the result as a hierarchical traffic-light tree view.

This example starts with an Excel file. The first column is used for determining which database table the row is stored in, with references to other row types above.

Excel is a very good tool for manual bulk editing and for viewing current states of complicated systems. It is one of many ways to input data into Nodelab.
When calculating status, a control can look into any other data source, both automatically collected or manually edited.

This Excel requirement definition file can either be added to a file structure that is part of the Nodelab resources, or it can be periodically or manually downloaded to a specific file directory.

Class

First, we define different clases for the rows. These classes are used by the server for automatically creating or modifying corresponding database tables. They are also used by the server and the browser in Edgescript® code.
This is the Requirement class.

class Requirement {
  @propertyTitle('Requirement Code')
  let reqCode: Char<255>

  @propertyTitle('Order Id')
  let orderId: Int

  @propertyTitle('Name')
  let name: Char<255>

  @propertyTitle('Review Time')
  let reviewTime: Datetime

  @propertyTitle('Description')
  let description: Text

  @propertyTitle('Comment')
  let comment: Text
}

Pattern

This is an Edgescript® example of a pattern parsing definition for the Excel file above.

    RecordPattern(
        isResource: true,
        pattern: 'Requirement(?:-.*)\.xlsx',
        processRecord: (storage: RecordStorage, record: [String], pathData: {String: Node}, env: {String: Node}): Boolean -> {
            let std = ReqStandard()
            let version = ReqVersion()
            let req = Requirement()
            var rowType, stdOrg, parentOrg, parentId, ctrlId: String
            RecordPattern.parse(record, \.rowType, \.stdOrg, \.std.stdId, \.version.versionId, \.version.effectiveFrom,
                \.req.reqCode, \.req.name, \.parentOrg, \.parentId, \.req.reviewTime, \.ctrlId, \.req.description,
                \.req.comment)
            if (rowType=='STD') {
                std._domId = env['domId']
                std.name = std.stdId
                storage.store(std)
                env['std'] = std
            } else if (rowType=='VER') {
                version._domId = env['domId']
                version.standard = env['std']
                version.name = version.versionId
                storage.store(version)
                env['version'] = version
            } else if (rowType=='REQ') {
                req._domId = env['domId']
                req.orderId = if env['orderId'] Int(env['orderId']) + 1 else 1
                storage.store(req)
                let content = ReqContent()
                content._domId = env['domId']
                content.version = env['version']
                content.req = req
                storage.store(content)
                env['req'] = req
                env['orderId'] = req.orderId
                if (parentId) {
                  let parent = Requirement()
                  parent.reqCode = parentId
                  let link = ReqParent()
                  link._domId = env['domId']
                  link.parent = parent
                  link.child = req
                  storage.store(link)
                }
            } else if (rowType=='CTRL') {
                let ctrl = Control()
                ctrl.ctrlId = ctrlId
                let link = ReqControl()
                link._domId = env['domId']
                link.req = env['req']
                link.ctrl = ctrl
                storage.store(link)
            }
            return true
        })

Evaluation

The script associated with each control is evaluated, periodically or at request, to produce status points for each control or configurable item, depending on the nature of the control.

Visualization

This is an example of code for a browser page. It looks similar to SwiftUI.
Note that the tree view uses a single query instance for the collection of a whole tree consisting of data from eleven different database tables, where each table can have its own parent-child dependency.
The status category, which determines the traffic-light color of each node, is aggregated from children to parents.

class ComplianceDashboard: Page {
  state layer = @pathClass(StatusLayer) QueryState(valuesQuery: Query(
    scripts: ['\.layerId/', '\.name/'],
    orderScripts: ['\.layerId/'],
    limit: 1000,
    isDistinct: true
  ))
  state reqCode = State()
  state req = @pathClass(Requirement) QueryState(Query(
    scripts: ['\.name/', '\.description/'],
    filters: ['\.reqCode/==\(reqCode)'],
    limit: 1
  ))
  init() {
    let treeQuery = @pathClass(ReqStandard) Query(
      titleScript: '\.name/',
      scripts: ['\.stdId/'],
      orderScripts: ['\.name/'],
      levels: [(
        parent: [(
          from: \ReqVersion.standard,
          to: \ReqStandard
        )],
        query: @pathClass(ReqVersion) Query(
          titleScript: '\.name/',
          scripts: ['\.versionId/'],
          orderScripts: ['\.name/'],
          maxClassCount: 2,
          levels: [(
            parent: [(
              from: \Requirement,
              to: \ReqContent.req
            ), (
              from: \ReqContent.version,
              to: \ReqVersion
            )],
            query: @pathClass(Requirement) Query(
              titleScript: '\.name/',
              scripts: ['\.reqCode/'],
              orderScripts: ['\.orderId/'],
              limit: 100000,
              maxClassCount: 5,
              parent: [(
                from: \Requirement,
                to: \ReqParent.child
              ), (
                from: \ReqParent.parent,
                to: \Requirement
              )],
              levels: [(
                parent: [(
                  from: \Control,
                  to: \ReqControl.ctrl
                ), (
                  from: \ReqControl.req,
                  to: \Requirement
                )],
                query:
                @pathClass(Control)
                @classOptions([
                  ClassOption(Status, name: 'status', filters: ['\Control/==\status.ctrl/']),
                  ClassOption(StatusCategory, name: 'statusCat', filters: ['\status.category/==\statusCat/'])
                ])
                Query(
                  titleScript: '\.name/',
                  scripts: ['\.ctrlId/', '\statusCat.level..max/'],
                  orderScripts: ['\.orderId/'],
                  limit: 100000,
                  maxClassCount: 12,
                  parent: [(
                    from: \Control,
                    to: \ControlParent.child
                  ), (
                    from: \ControlParent.parent,
                    to: \Control
                  )],
                  layers: [(
                    from: \StatusLayer,
                    to: \StatusLayer.parent
                  )],
                  layerPaths: [(
                    from: \StatusLayer,
                    to: \status.layer
                  )]
                )
              )]
            )
           )]
         )
      )]
    )
    super(
      VStack(
        HStack(
          StateSelector(\.layer, title: 'Layer', titleScript: '\.name/', defaultSelectionIndex: 0, css: 'col-md-2')
        ),
        HStack(
          TreeView(
            title: 'Standards',
            stateIcons: ['status-none', 'status-valid', 'status-partial', 'status-pending', 'status-invalid'],
            stateScript: '\StatusCategory.level/|0',
            query: treeQuery,
            css: 'col-md-4'
          )
        )
      )
    )
  }
}

UNQL

The query is translated into a UNQL request:

{
  "siteId": "ref",
  "packId": "ref.test",
  "appCode": "analytics",
  "domId": "org.test",
  "limit": 100,
  "joinBranches": 1,
  "query": {
    "levels": [
      {
        "parent": [
          {
            "from": "\\ReqVersion.standard",
            "to": "\\ReqStandard"
          }
        ],
        "query": {
          "levels": [
            {
              "parent": [
                {
                  "from": "\\Requirement",
                  "to": "\\ReqContent.req"
                },
                {
                  "from": "\\ReqContent.version",
                  "to": "\\ReqVersion"
                }
              ],
              "query": {
                "levels": [
                  {
                    "parent": [
                      {
                        "from": "\\Control",
                        "to": "\\ReqControl.ctrl"
                      },
                      {
                        "from": "\\ReqControl.req",
                        "to": "\\Requirement"
                      }
                    ],
                    "query": {
                      "pathClassName": "Control",
                      "classOptions": [
                        {
                          "name": "status",
                          "pathClassName": "Status",
                          "filters": [
                            "\\Control/==\\status.ctrl/"
                          ]
                        },
                        {
                          "name": "statusCat",
                          "pathClassName": "StatusCategory",
                          "filters": [
                            "\\status.category/==\\statusCat/"
                          ]
                        }
                      ],
                      "parent": [
                        {
                          "from": "\\Control",
                          "to": "\\ControlParent.child"
                        },
                        {
                          "from": "\\ControlParent.parent",
                          "to": "\\Control"
                        }
                      ],
                      "layers": [
                        {
                          "from": "\\StatusLayer",
                          "to": "\\StatusLayer.parent"
                        }
                      ],
                      "layerPaths": [
                        {
                          "from": "\\StatusLayer",
                          "to": "\\status.layer"
                        }
                      ],
                      "titleScript": "\\.name/",
                      "scripts": [
                        "\\.ctrlId/",
                        "\\statusCat.level..max/"
                      ],
                      "orderScripts": [
                        "\\.orderId/"
                      ],
                      "limit": 100000,
                      "maxClassCount": 12,
                      "validity": null
                    }
                  }
                ],
                "pathClassName": "Requirement",
                "classOptions": [],
                "parent": [
                  {
                    "from": "\\Requirement",
                    "to": "\\ReqParent.child"
                  },
                  {
                    "from": "\\ReqParent.parent",
                    "to": "\\Requirement"
                  }
                ],
                "titleScript": "\\.name/",
                "scripts": [
                  "\\.reqCode/"
                ],
                "orderScripts": [
                  "\\.orderId/"
                ],
                "limit": 100000,
                "maxClassCount": 5,
                "validity": null
              }
            }
          ],
          "pathClassName": "ReqVersion",
          "classOptions": [],
          "titleScript": "\\.name/",
          "scripts": [
            "\\.versionId/"
          ],
          "orderScripts": [
            "\\.name/"
          ],
          "limit": 100,
          "maxClassCount": 2,
          "validity": null
        }
      }
    ],
    "pathClassName": "ReqStandard",
    "classOptions": [],
    "titleScript": "\\.name/",
    "scripts": [
      "\\.stdId/"
    ],
    "orderScripts": [
      "\\.name/"
    ],
    "limit": 100,
    "maxClassCount": 1,
    "validity": null
  },
  "lang": "en"
}

SQL

The UNQL request is translated into four SQL queries.

SELECT T1.NAME AS T1_NAME,
	T1.stdId AS T1_stdId
FROM ORG_TEST_REF_TEST_REQSTANDARD T1
ORDER BY T1.NAME

SELECT T1.stdId AS T1_stdId,
	T2.NAME AS T2_NAME,
	T2.versionId AS T2_versionId,
	T2.STANDARD_stdId AS T2_STANDARD_stdId,
	T2.STANDARD_stdId AS T2_STANDARD_stdId
FROM ORG_TEST_REF_TEST_REQSTANDARD T1
LEFT JOIN ORG_TEST_REF_TEST_REQVERSION T2 ON (T1.stdId = T2.STANDARD_stdId)
ORDER BY T2.NAME

SELECT T2.versionId AS T2_versionId,
	T2.STANDARD_stdId AS T2_STANDARD_stdId,
	T3.REQ_REQCODE AS T3_REQ_REQCODE,
	T3.VERSION_versionId AS T3_VERSION_versionId,
	T3.VERSION_VERSION_stdId AS T3_VERSION_VERSION_stdId,
	T4.NAME AS T4_NAME,
	T4.ORDERID AS T4_ORDERID,
	T4.REQCODE AS T4_REQCODE,
	T4.REQCODE AS T4_REQCODE,
	T4.REQCODE AS T4_REQCODE,
	T5.CHILD_REQCODE AS T5_CHILD_REQCODE,
	T5.PARENT_REQCODE AS T5_PARENT_REQCODE
FROM ORG_TEST_REF_TEST_REQSTANDARD T1
LEFT JOIN ORG_TEST_REF_TEST_REQVERSION T2 ON (T1.stdId = T2.STANDARD_stdId)
LEFT JOIN ORG_TEST_REF_TEST_REQCONTENT T3 ON (T2.STANDARD_stdId = T3.VERSION_VERSION_stdId)
AND (T2.versionId = T3.VERSION_versionId)
LEFT JOIN ORG_TEST_REF_TEST_REQUIREMENT T4 ON (T3.REQ_REQCODE = T4.REQCODE)
LEFT JOIN ORG_TEST_REF_TEST_REQPARENT T5 ON (T5.CHILD_REQCODE = T4.REQCODE)
ORDER BY T4.ORDERID

SELECT T4.REQCODE AS T4_REQCODE,
	T6.CTRL_ctrlId AS T6_CTRL_ctrlId,
	T6.REQ_REQCODE AS T6_REQ_REQCODE,
	T7.ctrlId AS T7_ctrlId,
	T7.ctrlId AS T7_ctrlId,
	T7.NAME AS T7_NAME,
	T7.ORDERID AS T7_ORDERID,
	MAX(STATUSCAT_T8.LEVEL) AS STATUSCAT_T8_LEVEL,
	T9.CHILD_ctrlId AS T9_CHILD_ctrlId,
	T9.PARENT_ctrlId AS T9_PARENT_ctrlId,
	T10.LAYERID AS T10_LAYERID,
	STATUS_T11.LAYER_LAYERID AS STATUS_T11_LAYER_LAYERID
FROM ORG_TEST_REF_TEST_REQSTANDARD T1
LEFT JOIN ORG_TEST_REF_TEST_REQVERSION T2 ON (T1.stdId = T2.STANDARD_stdId)
LEFT JOIN ORG_TEST_REF_TEST_REQCONTENT T3 ON (T2.STANDARD_stdId = T3.VERSION_VERSION_stdId)
AND (T2.versionId = T3.VERSION_versionId)
LEFT JOIN ORG_TEST_REF_TEST_REQUIREMENT T4 ON (T3.REQ_REQCODE = T4.REQCODE)
LEFT JOIN ORG_TEST_REF_TEST_REQPARENT T5 ON (T5.CHILD_REQCODE = T4.REQCODE)
LEFT JOIN ORG_TEST_REF_TEST_REQCONTROL T6 ON (T4.REQCODE = T6.REQ_REQCODE)
LEFT JOIN ORG_TEST_REF_TEST_CONTROL T7 ON (T6.CTRL_ctrlId = T7.ctrlId)
LEFT JOIN ORG_TEST_REF_TEST_CONTROLPARENT T9 ON (T9.CHILD_ctrlId = T7.ctrlId)
LEFT JOIN ORG_TEST_REF_TEST_STATUS STATUS_T11 ON (T7.ctrlId = STATUS_T11.CTRL_ctrlId)
LEFT JOIN ORG_TEST_REF_TEST_STATUSLAYER T10 ON (STATUS_T11.LAYER_LAYERID = T10.LAYERID)
LEFT JOIN ORG_TEST_REF_TEST_STATUSCATEGORY STATUSCAT_T8 ON (STATUS_T11.CATEGORY_NAME = STATUSCAT_T8.NAME)
GROUP BY STATUS_T11_LAYER_LAYERID,
	T10_LAYERID,
	T4_REQCODE,
	T6_CTRL_ctrlId,
	T6_REQ_REQCODE,
	T7_ctrlId,
	T7_ctrlId,
	T7_NAME,
	T7_ORDERID,
	T9_CHILD_ctrlId,
	T9_PARENT_ctrlId
ORDER BY T7.ORDERID

The results are combined into a tree structure, and the server returns a JSON tree structure based on data from eleven different database tables.