Parsing Simulink mdl files with Pyparsing

• Published 2007-05-14 (12 months ago)
• Updated 2007-08-27 (8 months, 2 weeks ago)

I am currently working on a small side project where I need to extract information from Simulink mdl files. Thanks to the Pyparsing module, it is surprisingly easy to create a full-fledged parser. I am not aware of any publicly available Simulink parsers, so I am posting the full source code here. Maybe someone will find it useful.

The Simulink file format

The file format used to store Simulink models is really simple. The data is stored in a structured ASCII file. Below are excerpts from a mdl file.

System {
    Name		    "feedbackloop"
    Location		    [480, 85, 1016, 386]
    PaperType		    "A4"
    TiledPaperMargins	    [0.500000, 0.500000, 0.500000, 0.500000]
    ...
    Block {
      BlockType		      Fcn
      Name		      "Fcn"
      Position		      [120, 50, 180, 80]
      Expr		      "sin(u(1)*exp(2.3*(-u(2))))+sin(u(1)*exp(2.3*(-u"
"(2))))+sin(u(1)*exp(2.3*(-u(2))))+sin(u(1)*exp(2.3*(-u(2))))+sin(u(1)*exp(2.3"
"*(-u(2))))+sin(u(1)*exp(2.3*(-u(2))))+sin(u(1)*exp(2.3*(-u(2))))"
    }
    ...
    Line {
      SrcBlock		      "Integrator"
      SrcPort		      1
      Points		      [35, 0]
      Branch {
	Points			[0, -45; 40, 0]
	DstBlock		"Out1"
	DstPort			1
      }
      Branch {
	Points			[0, 70; -25, 0]
	Branch {
	  Points		  [-95, 0]
	  DstBlock		  "Sum"
	  DstPort		  2
	}
	Branch {
	  Points		  [0, 40; -135, 0]
	}
    ...
      }
    }
}

A few observations:

• The data is hierarchical.
• Parameters and values are separated by whitespace
• Long strings are split over multiple lines
• Object parameter types are strings, integers, floats, lists and matrices

Although the Simulink file format is simple, parsing the data is not trivial due to the nested objects. Writing a parser from scratch is time consuming. Fortunately there exists a Python library that provides the necessary boiling-plate code and does most of the hard job.

Pyparsing to the rescue

Trying Pyparsing has been on my todo-list for a long time, and the need for a Simulink parser was a perfect opportunity to get acquainted with Pyparsing. The need for parsing data structures arises from time to time, and Pyparsing seems to be the right tool for the job. I also have a very good impression of the author, Paul McGuire. He is a frequent contributor to comp.lang.python and always has helpful and insightful comments.

My standard approach when learning a new tool, is to try find an example that is tangential to a problem I want to solve. I then tweak and adapt the example to get a hands-on feeling of how things work. In this case I found an example that is almost a perfect fit: jsonParser.py. The example shows how to parse the JSON format, which has many similarities with the Simulink file format. After some experimentation I was able to adapt the example to parse Simulink data.

Code

This is my first attempt to write a parser using Pyparsing, so the quality of the code may not be the best. You can also download the code directly: simulinkparser.py

Update: Fixed problem with strings split over multiple lines. Latest version of Pyparsing broke the code.

"""
A simple Simulink mdl file parser.

Credits:
Most of the code is based on the json parser example distributed with
pyparsing. The code in jsonParser.py was written by Paul McGuire


"""

__author__ = 'Kjell Magne Fauske'
__license__ = 'MIT'

# A high level grammar of the Simulink mdl file format
SIMULINK_BNF = """
object {
     members
}
members
    variablename  value
    object {
        members
    }
variablename

array
    [ elements ]
matrix
    [elements ; elements]
elements
    value
    elements , value
value
    string
    doublequotedstring
    float
    integer
    object
    array
    matrix
"""

from pyparsing import *

# parse actions
def convertNumbers(s,l,toks):
    """Convert tokens to int or float"""
    # Taken from jsonParser.py
    n = toks[0]
    try:
        return int(n)
    except ValueError, ve:
        return float(n)

def joinStrings(s,l,toks):
    """Join string split over multiple lines"""
    return ["".join(toks)]

# Define grammar
dblString = dblQuotedString.setParseAction( removeQuotes )
mdlNumber = Combine( Optional('-') + ( '0' | Word('123456789',nums) ) +
                    Optional( '.' + Word(nums) ) +
                    Optional( Word('eE',exact=1) + Word(nums+'+-',nums) ) )
mdlObject = Forward()
mdlName = Word('$'+'.'+alphas+nums)
mdlValue = Forward()
# Strings can be split over multiple lines
mdlString = (dblString + Optional(OneOrMore(Suppress(LineEnd()) + LineStart()
             + dblString)))
mdlElements = delimitedList( mdlValue )
mdlArray = Group(Suppress('[') + Optional(mdlElements) + Suppress(']') )
mdlMatrix =Group(Suppress('[') + (delimitedList(Group(mdlElements),';')) \
              + Suppress(']') )
mdlValue << ( mdlNumber | mdlName| mdlString  | mdlArray | mdlMatrix )
memberDef = Group( mdlName  + mdlValue ) | Group(mdlObject)
mdlMembers = OneOrMore( memberDef)
mdlObject << ( mdlName+Suppress('{') + Optional(mdlMembers) + Suppress('}') )
mdlNumber.setParseAction( convertNumbers )
mdlString.setParseAction(joinStrings)
mdlparser = mdlObject

if __name__ == '__main__':
    import pprint
    testdata = """
    System {
        Name		    "feedbackloop"
        Location		    [480, 85, 1016, 386]
        Open		    on
        TiledPaperMargins	    [0.500000, 0.500000, 0.500000, 0.500000]
        Block {
            BlockType    Fcn
            Name	     "Fcn"
            Position     [120, 50, 180, 80]
            Expr	     "sin(u(1)*exp(2.3*(-u(2))))+sin(u(1)*exp(2.3*(-u"
"(2))))+sin(u(1))"
        }
        Block {
          BlockType		      Outport
          Name		      "Out1"
          Position		      [345, 73, 375, 87]
          IconDisplay	      "Port number"
          BusOutputAsStruct	      off
        }
        Line {
          SrcBlock		      "Integrator"
          SrcPort		      1
          Points		      [35, 0]
          Branch {
    	      Points			[0, -45; 40, 0]
    	      DstBlock		"Out1"
    	      DstPort			1
          }
        }
    }
    """
    result = mdlparser.parseString(testdata)
    pprint.pprint(result.asList())

Example output:

['System',
 ['Name', 'feedbackloop'],
 ['Location', [480, 85, 1016, 386]],
 ['Open', 'on'],
 ['TiledPaperMargins', [0.5, 0.5, 0.5, 0.5]],
 ['Block',
  ['BlockType', 'Fcn'],
  ['Name', 'Fcn'],
  ['Position', [120, 50, 180, 80]],
  ['Expr', 'sin(u(1)*exp(2.3*(-u(2))))+sin(u(1)*exp(2.3*(-u(2))))+sin(u(1))']],
 ['Block',
  ['BlockType', 'Outport'],
  ['Name', 'Out1'],
  ['Position', [345, 73, 375, 87]],
  ['IconDisplay', 'Port number'],
  ['BusOutputAsStruct', 'off']],
 ['Line',
  ['SrcBlock', 'Integrator'],
  ['SrcPort', 1],
  ['Points', [35, 0]],
  ['Branch',
   ['Points', [[0, -45], [40, 0]]],
   ['DstBlock', 'Out1'],
   ['DstPort', 1]]]]

Final words

Writing a full-fledged parser with Pyparsing is so easy that it almost feels like cheating. However, writing the parser is only the first step. The next thing I have to do is to find a practical and efficient way of using the parsed data.

The output from the parser is a nested list structure. This structure is a bit impractical if I for instance quickly want to find out if two blocks are connected. Pyparsing offers a few hooks to manipulate data during parsing, like for instance parse actions. The easiest solution is probably to iterate a few times over the output and store the data I need in an appropriate data structure.