Wednesday Night Hack #2 - BARF

Tonight, I continued to work on the front end tool described in last week's blog post: http://blog.edmondcote.com/2014/05/wednesday-hack-1.html.  I found a clever name for the script: barf (build and run flow).  It's too late in the evening to post details, so I will be brief.  Besides the name change, I implemented an object oriented build pipeline.  The user can describe the stages of their build and run flow using Python objects.  The execution of the objects is controlled by a centralized class.  While it may be outside of the scope of this (yet to be determined effort), this would allow the jobs to be executed in parallel.  I also added a mechanism for a job stage to pass information from one another.  An example use case is for the synthesis step to pass an object to the place and route step.  Finally, I created two build stages.  The first to create an Active-HDL work library (vlib) and the second to compile Verilog file (vlib).

Wednesday Night Hack #1

This week, I began the development of basic a front end tool to wrap the functionality of the CAD tools bundled with my LatticeECP3 FPGA development board.  The packaged tools are Aldec-HDL, Synplify Pro, and Lattice Diamond (the FPGA implementation tool).  Active-HDL, by all respects, have a pretty solid GUI, but my preference is to maintain control from the command line.

Rather than implement a custom Makefile library or a traditional Perl script, I chose to develop an API using Python that a user could use to build up their own flow apps.  The idea is to replace configuration files or application-specific dynamic scripting languages with an actual scripting language.  This approach scales better over time.

The first step to this process is project management. We want the ability to manage groups of files.  I call these groups of files components.  Examples of components include RTL unit (Verilog module and its sub modules), UVC (Universal Verification Component), design IP library (RAM library, FPGA megafunctions), etc.

We also want the ability to establish "depends on" relationships between components.  Here is an example component definition written in Python

c.set_name('top')
c.add_file('top.v')
c.add_require('led')
c.add_require('ram')

For a simple project consisting of three components. Top, RAM, and LED where top depends on RAM and LED and LED also depends on RAM.  The flist would need to resemble following.  The script below is able to produce this.  In fact, below is the actual output of the script.

# Component: ram
/cygdrive/d/Projects/system/rtl/ram/ram.v

# Component: led
# Requires: ram
/cygdrive/d/Projects/system/rtl/led/led.v

# Component: top
# Requires: led,ram
/cygdrive/d/Projects/system/rtl/top/top.v

I achieve this functionality by implementing two classes in Python: Component and Go.  Here are some code snippets.  First, I build a tree data structure in process_requires function.  I traverse the tree in function get_flist using a simple recursive algorithm.

Here is a snippet of the source code.  There's no error checking, so YMMV.

class Component:
    """ Exposed to user """
    def add_file(self, file):
        self.files.append(self.root_dir+'/'+file)

    """ Exposed to user """
    def add_option(self, option):
        self.options.append(option)

    """ Exposed to user """
    def add_require(self, require):
        self.requires.append(require)

    def get_flist(self):
        flist = '#' * 80 + '\n' # 80 character comment line
        flist +=  '# Component: ' + self.get_name() + '\n'
        if self.requires:
            flist +=  '# Requires: '+','.join(self.requires)+'\n'
        if self.files:
            flist +=  '\n'.join(self.files) + '\n'
        if self.options:
            flist +=  '\n'.join(self.options) + '\n'
        return flist

class Go:
    """ Load all component under root_dir """
    def load_comps(self):
        for root,dirs,files in os.walk(self.root_dir):
            for f in files:
                if f == 'comp.py':
                    c = Component()
                    c.set_root_dir(root)
                    execfile(root+'/'+f)
                    self.component[c.get_name()] = c

    def process_requires(self,node):
        for require in node.requires:
            child_node = self.component[require]
            self.process_requires(child_node)
            node.child_nodes.append(child_node)

    def init(self):
        self.load_comps()
        self.process_requires(self.get_top_node())

    def get_flist(self):
        self.visited = {}
        self.do_get_flist(self.get_top_node())

    """ Traverse tree using recursive post-order search """
    def do_get_flist(self,node):
        for child in node.child_nodes:
            if not child.get_name() in self.visited:
                self.do_get_flist(child)
        if not node.get_name() in self.visited:
           print(node.get_flist())
           self.visited[node.get_name()] = 1 # mark visited

Back to Basics

I have been working in hardware design verification since I graduated from school.  It has been nearly 7 years now, time flies, I can hardly believe it.

Hardware design verification is an extremely exciting field.  The field requires knowledge spanning multiple disciplines: software development, hardware implementation, and (in my case) computer/graphics systems.

My job at Qualcomm is also extremely challenging, we are at the forefront of all the above.

At school, I wasn't interested so much in academics or research as I was building things.  I spent a substantial amount of time working on overly ambitious projects.

The best example of that, perhaps, was my digital VLSI circuits class where I spent more time playing with tools to generate GDS-II from RTL than focusing on the class assignment which was to design and layout a custom digital cell.

In my undergraduate computer graphics class, I went way beyond the defacto rendered-with-OpenGL textured cube or simple ray tracer. I wrote my own scene graph, associated rendering engine, and custom Python-based Blender3D export plugin.  Not only that, but I took it a step further and wrote some custom shaders in ARB assembly language.  This was before there were high-level shader languages.

Fast forward to grad school.  I had taken a keen interest in computer architecture.  The thought and opportunity to understand these systems from the ground up was (and still is) very exciting for me.

As you can imagine, I wasn't interested so much in research as I was in hacking.  I spent a substantial amount of time building my own multiprocessor system-on-a-chip from the ground up and prototyped it on an FPGA.

The final product was documented in my thesis and in two conference papers.  Granted, my contributions weren't ground breaking, but in retrospec, it was a very rewarding experience.

Over the past few years, I haven't tinkered so much. I worked. I lived. I got married. I am still learning about work-life balance. But, the bug to tinker is still in me.

This long winded rant to say that I have recently been researching FPGA prototype boards to purchase and settled on this one the other day.  Besides cost, I was motivated by the availability of Ethernet ports and PCIe connector.  I/O is an area of system design that I know next to nothing about.

http://www.latticesemi.com/en/Products/DevelopmentBoardsAndKits/LatticeECP3VersaDevelopmentKit.aspx

I do not plan to make any commitments regarding my tinkering, but it is a part of me that I hope to resurrect.

Vim Setup for Python - Part 2

A quick follow up of my previous post:

http://blog.edmondcote.com/2013/10/vim-setup-for-python.html

Tagbar install and shortcut

1. Install ctags
2. Download Tagbar from http://www.vim.org/scripts/download_script.php?src_id=21362
3. Install Tagbar using vim -c 'so %' -c 'q' tagbar.vmb
4. Add nmap <F8> :TagbarToggle<CR> to your $HOME/.vimrc

Shortcut to fix indentation

http://vim.wikia.com/wiki/Fix_indentation

1. Add map <F7> mzgg=G`z<CR> to your $HOME/.vimrc

Tab-completion of Python code

1. Download latest version of Pydiction http://www.vim.org/scripts/script.php?script_id=850
2. Install Pydiction

Vim Setup for Python

I have used a number of modern IDEs for software development.  For example, I think it is neat how one can draw up UML diagrams and generate code from them (or vice versa).  Integration with the-more-complex revision control systems is also nice.  What I dislike is vendor lock in loss of control of the development flow.  I could speak more about this, but it is a different topic than what I'd like to cover today.

I like the Vim editor because of it's point tool nature.  It does it's job and it does it well.  It is well supported by the community, it is available on virtually every Linux installation (and Windows), and it is extremely customization.

A number of blog posts have been written that describe their settings for configuring Vim to edit Python files. This will be yet another.  The default configuration for Python in Vim leaves to be desired.  Fortunately, there are a few easy steps one can take to enable a streamlined experience.

The first step is to make sure to enable the Vim filetype plugin using command filetype plugin indent on in your $HOME/.vimrc.  The following site has more information about why to use this plugin.  To summarize, it allows the user to move filetype-specific commands outside of their main configuration file, reducing clutter in the file.

http://vim.wikia.com/wiki/Keep_your_vimrc_file_clean

For indentation, first create a Python-specific configuration file $HOME/.vim/ftplugin/python.vim containing the following lines:

setlocal tabstop=4

setlocal softtabstop=4

setlocal shiftwidth=4

setlocal textwidth=79 " to comply with PEP-8 style guide

setlocal smarttab

setlocal expandtab

Eric McSween developed a Vim plugin for smart indentation of  Python code.  The utility is available here: http://www.vim.org/scripts/script.php?script_id=974.  Installation is straightforward, simply download the latest version of the script and copy the file to $HOME/.vim/indent/python.vim.  Use of the filetype plugin, as described above, will automatically enable the utility.

Last, Dmitry Vasiliev developed a Vim plugin for Python syntax highlighting.  The utility is available here: http://www.vim.org/scripts/script.php?script_id=790.  To install the plugin, download the latest version of the script and copy the file to $HOME/.vim/syntax/python.vim.  Enable the plugin by adding command syntax on to your $HOME/vim.rc file.

I recommend the use of a static code analyzer for anyone developing new code.  Pylint is my analysis tool of choice.  It is typically available as a binary package in most Linux distros - i.e. may need to talk to your sys admin.  There is some overhead of using code analysis in the short term, but long term, you will benefit from a code base with that looks pretty and smells less.

For integration with my editor, I use the following plugin: http://www.vim.org/scripts/script.php?script_id=891.  Installation of the plugin and requires the user to copy the downloaded script to directory $HOME/.vim/compiler.