Grammars and Parsing

In order to process arbitrarily formated textual information, we resort to EBNF grammars and parsing. To achieve that, we depend on Lark. The structure of the grammar directory is the following. Each *.lark file nests a Lark-compliant grammar. We differentiate between two types of grammars.

1. Grammars related to segments of the textual fault report (rpt) of zoix, which are prefixed with frpt_

2. Grammars related to traces, which are prefixed with trace_

Each grammar must have its corresponding Transformer to traverse the AST and generate data structures. Specifically for the trace related grammars, what is expected from the transformer is to transform the AST into a CSV-ready format as a list of strings.

Whenever a new grammar is added with its corresponding transformer in the transformers.py the corresponding factories mush be updated accordingly.

class grammars.transformers.FaultReportCoverageTransformer(visit_tokens: bool = True)

Bases: Transformer

This transformer is expected to act on the grammar of the Coverage segment of a Z01X txt fault report.

It constructs and returns a dictionary with keys the formula names and keys the corresponding formulas as strings.

formula(formula_name: str, formula: str) → tuple[str, str]

Takes the lhs and rhs and returns them as a tuple .. highlight:: python .. code-block:: python

“Diagnostic Coverage” = “INT(DD/(NA + DA + DN + DD))”;

^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^

consumed consumed

lhs(formula_name: str) → str

Digests the left hand side of a single formula and returns the formula name as string.

"Diagnostic Coverage" = "INT(DD/(NA + DA + DN + DD))";
 ^^^^^^^^^^^^^^^^^^^
      consumed

rhs(formula: str) → str

Digests the right hand side of a single formula while skipping format specifiers (if any). Returns the formula as string.

"Diagnostic Coverage" = "INT(DD/(NA + DA + DN + DD))";
                            ^^^^^^^^^^^^^^^^^^^^^^^^
                                consumed

start(formulas: list[tuple[str, str]]) → dict[str, str]

Accepts a series of formulas in the form of tuples and transforms them into a dictionary.

class grammars.transformers.FaultReportFaultListTransformer(visit_tokens: bool = True)

Bases: Transformer

This transformer is expected to act on the grammar of the FaultList segment of a Z01X txt fault report.

After parsing the segment is returning a list of zoix.Fault objects with the following attributes:

• fault_status (str): 2-uppercase-letter status

• fault_type (str): 0|1|R|F|~

• timing_info (list[str]): A list with all timing info (if present) e.g., [‘6.532ns’]

• fault_sites (list[str]): A list of fault sites represented as strings

• fault_attributes (dict[str, str]): A dictionary with all fault attributes (if present)

Lastly, it resolves on-the-fly any fault equivalences on the generated fault list.

attribute_and_value(attribute_name: str, attribute_value: str) → tuple[str, str]

Provides a single attribute (if present) of a prime fault as a tuple. The tuple holds the attribute name and the corresponding attribute value.

< 1 1 1> ON 1 { PORT "tb_top.dut.subunit_a.cell.port" } (* "testname"->attr_a=12345; "testname"->attr_b=0xA)
                                                            ^^^^^^^^   ^^^^^^ ^^^^^
                                                            ignored      consumed

attributes(attributes: list[tuple[str, str]]) → tuple[Literal['Fault Attributes'], dict[str, str]]

Provides attributes (if present) of a prime fault as a list of tuples. Each tuple holds the attribute name and the corresponding attribute value.

< 1 1 1> ON 1 { PORT "tb_top.dut.subunit_a.cell.port" } (* "testname"->attr_a=12345; "testname"->attr_b=0xA)
                                                            ^^^^^^^^   ^^^^^^ ^^^^^   ^^^^^^^^  ^^^^^^^ ^^^
                                                            ignored      consumed      ignored    consumed

fault(fault_parts: list[tuple[str, Any]]) → Fault

Returns a Fault object for each line in the FaultList section.

In this part of the AST all fault information has been parsed and all information needed to represent a fault is available.

< 1 1 1> ON 1 { PORT "tb_top.dut.subunit_a.cell.port" } (* "test"->attr=val; *)
         |   \       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^               ^^^^^^^^
Fault Status  \      Fault Sites (list)               Fault Attributes (dict)
             Fault Type

Parameters:

fault_parts (list) – A list of tuples which hold all information needed to represent a fault.

Returns:

A Fault object whose attributes are: - Fault_Status: str - Fault_Type: str - Fault_Sites: list[str] - Fault_Attributes: dict[str, str]

Return type:

Fault

fault_info(args) → _DiscardType

Consumes the fault info segment of a line (if present) and discards it.

< 1 1 1> ON 1 { PORT "tb_top.dut.subunit_a.cell.port" }
 ^^^^^^
discarded

fault_status(fault_status: str) → tuple[Literal['Fault Status'], str]

Consumes the 2-letter fault status attribute of a line.

< 1 1 1> ON 1 { PORT "tb_top.dut.subunit_a.cell.port" }
         ^^
      consumed

fault_type(fault_type: str) → str

Provides the fault type attribute of a line.

< 1 1 1> ON 1 { PORT "tb_top.dut.subunit_a.cell.port" }
            ^
         consumed

static filter_out_discards(container: Iterable) → filter

loc_and_site(fault_site: str) → str

Provides the fault site’s hierarhical path from each parsed line.

< 1 1 1> ON 1 { PORT "tb_top.dut.subunit_a.cell.port" }
                ^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               ignored       consumed

location_info(sites: list[str]) → tuple[Literal['Fault Sites'], list[str]]

Provides the fault site’s hierarhical path from each parsed line as a list of strings.

< 1 1 1> ON 1 { PORT "tb_top.dut.subunit_a.cell.port1" } + { PORT "tb_top.dut.subunit_a.cell.port2" }
                ^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^        ^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               ignored       consumed                      ignored       consumed

optional_name(fault_list_name: str) → _DiscardType

Discard the name of the fault list.

FaultList SomeCNAMEfaultListName {
          ^^^^^^^^^^^^^^^^^^^^^^
               discarded

start(faults: list[Fault]) → list[Fault]

Parsing is finished. The fault list has been generated.

timing_info(timings: list[str]) → tuple[Literal['Timing Info'], list[str]]

Takes all timing info (if present) and returns it as a list of the string-ified tokens.

<  1> NN R (7.52ns) {FLOP "tb_top.dut.subunit_a.cell.port1"}
            ^^^^^^
            consumed

class grammars.transformers.FaultReportStatusGroupsTransformer(visit_tokens: bool = True)

Bases: Transformer

This transformer is expected to act on the grammar of the StatusGroups segment of a Z01X txt fault report.

It constructs and returns a dictionary with keys the group names and keys the statuses of each group.

fault_statuses(*statuses) → list[str]

Digests the fault statuses of a group and returns them as a list of strings

SA "Safe" (UT, UB, UR, UU);
           ^^  ^^  ^^  ^^
             captured

group(name: str, statuses: list[str]) → tuple[str, list[str]]

Digests group name and extended group name and receives the transformed statuses. Returns the group name and the statuses (as-is) in the form of a tuple

SA       "Safe" (UT, UB, UR, UU);
^^        ^^^^
captured  discarded

start(groups: list[tuple[str, list[str]]]) → dict[str, list[str]]

Receives all groups in the form of tuples where the index 0 is the group name and index 1 is a list of statuses. Returns a dictionary where each key is a group and the corresponding value is the related statuses as list[str].

class grammars.transformers.FaultReportTransformerFactory

Bases: object

Factory pattern for Z01X txt fault report transformers and the corresponding grammars.

To be used as:

factory = FaultReportTransformerFactory()
parser = factory("FaultReportSectionString")

class grammars.transformers.TraceTransformerCV32E40P(visit_tokens: bool = True)

Bases: Transformer

Transformer for the grammar of the tracer of CV32E40P.

When applied, returns the trace as a list of strings. The string at index 0 is the header and the rest the trace entries. Intended for converting the textual trace format to CSV.

More information about the CV32E40P tracer format here.

entries(time: Token, cycle: Token, pc: Token, instr: Token, decoded_instr: Token, reg_and_mem: Token | None = None) → str

Processes a single entry line and returns it as a csv-ready string.

142  67 0000015c c622 c.swsp  x8,12(x2) x2:0x00002000 x8:0x00000000 PA:0x0000200c store:0x0 load:0xffffffff
^^^  ^^ ^^^^^^^^ ^^^^ ^^^^^^  ^^^^^^^^^ ^^^^^^^^^^^^^ ^^^^^^^^^^^^^ ^^^^^^^^^^^^^ ^^^^^^^^^ ^^^^^^^^^^^^^^^
Time Cycle  PC   Instr Decoded Instr.   Register and memory contents

header(*fields: Token) → str

Handles the header line of the trace.

Time          Cycle      PC       Instr    Decoded instruction Register and memory contents
^^^^          ^^^^^      ^^       ^^^^^    ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
captured     captured  captured  captured  captured            captured

reg_and_mem(*reg_val_pairs: str) → str

Processes a single ‘Register and memory contents’ entry (if present in the trace line).

x2:0x00002000 x8:0x00000000 PA:0x0000200c store:0x0 load:0xffffffff
^^^^^^^^^^^^^ ^^^^^^^^^^^^^ ^^^^^^^^^^^^^ ^^^^^^^^^ ^^^^^^^^^^^^^^^

start(header_and_entries: list[str]) → list[str]

Parsing is finished. Accepts the trace as a list of strings and returns it as-is.

class grammars.transformers.TraceTransformerFactory

Bases: object

Factory pattern for trace transformers and the corresponding grammars.

To be used as:

factory = TraceTransformerFactory()
parser = factory("ProcessorString")