Decomposing TileLink2 in RocketChip: A Barebones System

Quick post.  I spent some time yesterday/today hacking at the TileLink2 implementation in RocketChip.

(As a matter of coincidence, SiFive recently released a draft spec of the protocol.)

My preferred method of learning is first to understand the engineer's original intent before enhancing or leveraging code.  This means taking things apart.

The most difficult aspect in getting something up was understanding the inner workings of SystemBus class.  I kept hitting run time assertions and it was helpful to manually sketch out the connections between the TL objects:

Here is a summary of my findings.

I identified the minimal set of configuration knobs for this experiment.

// Barebones system configuration
class BarebonesSystemConfig extends Config((site, here, up) => {
  case XLen => 64
  case BankedL2Params => BankedL2Params(nMemoryChannels = 0,
    nBanksPerChannel = 1,
    coherenceManager = { // FIXME: not really a coherence manager
      case (q, _) =>
        implicit val p = q
        val cork = LazyModule(new TLCacheCork(unsafe = true))
        (cork.node, cork.node)
    })
  case SystemBusParams => SystemBusParams(beatBytes = site(XLen) / 8, blockBytes = site(CacheBlockBytes))
  case PeripheryBusParams => PeripheryBusParams(beatBytes = site(XLen) / 8, blockBytes = site(CacheBlockBytes))
  case MemoryBusParams => MemoryBusParams(beatBytes = 8, blockBytes = site(CacheBlockBytes))
  case CacheBlockBytes => 64
  case DTSTimebase => BigInt(1000000) // 1 MHz
  case DTSModel => "bboneschip"
  case DTSCompat => Nil
  case TLMonitorBuilder => (args: TLMonitorArgs) => Some(LazyModule(new TLMonitor(args)))
  case TLCombinationalCheck => false
  case TLBusDelayProbability => 0.0
})

This is the system itself.  It extends BaseComplex.  Not RocketComplex.

class BarebonesSystem(implicit p: Parameters) extends BaseCoreplex {
  // connect single master device (fuzzer) to system bus (sbus)
  sbus.fromSyncPorts() :=* LazyModule(new TLFuzzer(1)).node

  // connect single slave device (memory) to the periphery bus (pbus)
  LazyModule(new TLTestRAM(AddressSet(0x0, 0xfff))).node :=* pbus.toFixedWidthSingleBeatSlave(4)

  override lazy val module = new BarebonesSystemModule(this)
}
class BarebonesSystemModule[T <: BarebonesSystem](_outer: T) extends BaseCoreplexModule(_outer) {}

This should be familiar.  It is your top level Chisel3 module.

// Chisel top module
class Top(implicit val p: Parameters) extends Module {
  val io = new Bundle {
    val success = Bool(OUTPUT)
  }
  val dut = Module(LazyModule(new BarebonesSystem).module)
  dut.reset := reset
}

Here are the important parts of the system generator.  This will generate your firrtl and verilog output.

// System generator
class System extends HasGeneratorUtilities {
  val names = new ParsedInputNames(
    targetDir = ".",
    topModuleProject = "com.bbones",
    topModuleClass = "Top",
    configProject = "",
    configs = "system"
  )

  val config = new BarebonesSystemConfig
  val params = Parameters.root(config.toInstance)
  val circuit = elaborate(names, params)

  def generateArtefacts: Unit =
    ElaborationArtefacts.files.foreach {
      case (extension, contents) =>
        writeOutputFile(names.targetDir, s"${names.configs}.${extension}", contents())
    }

  def generateFirrtl: Unit =
    Driver.dumpFirrtl(circuit, Some(new File(names.targetDir, s"${names.configs}.fir")))

  def generateVerilog: Unit = {
    import sys.process._
    val firrtl = "/home/edc/gitrepos/rocket-chip/firrtl/utils/bin/firrtl"
    val path = s"${names.targetDir}/${names.configs}"
    val bin = s"$firrtl -i $path.fir -o $path.v -X verilog"
    bin.!
  }

  generateArtefacts
  generateFirrtl
  generateVerilog
}

Finally, one should never forget unit testing!

// Unit test
class SystemSpec extends FlatSpec with Matchers {
  "An instance of the system generator " should "compile" in {
    val s = new System
  }
}

Gentle introduction to diplomacy

In a previous post, I was looking at how to use the diplomacy and uncore libraries (in Rocket Chip Generator) to hook up a simple AHB master module to AHB slave module.  To my knowledge, none of the library code is documented.  IDEA's debugger came in handy to grow my understanding of the code base.  I can confirm the wiring portion now works.  May come back to this in the future.

Master module

class Master()(implicit p: Parameters) extends LazyModule
{
  val ahbMasterParameters = AHBMasterParameters(
    // nothing for now ..
  )

  val ahbMasterPortParameters = AHBMasterPortParameters(
    masters = Seq(ahbMasterParameters)
  )

  val node = AHBMasterNode(
    portParams = Seq(ahbMasterPortParameters)
  )

  lazy val module = new LazyModuleImp(this) {
    val io = new Bundle {
      val out = node.bundleOut
    }
  }
}

Slave module

class Slave()(implicit p: Parameters) extends LazyModule
{
  val ahbSlaveParameters = AHBSlaveParameters(
    address = Seq(AddressSet(0x0, 0xFFFF)),
    regionType    = RegionType.UNCACHED,
    executable    = true,
    supportsRead  = TransferSizes(1, 4),
    supportsWrite = TransferSizes(1, 4)
  )

  val ahbSlavePortParameters = AHBSlavePortParameters(
    slaves = Seq(ahbSlaveParameters),
    beatBytes = 4
  )

  val node = AHBSlaveNode(
    portParams = Seq(ahbSlavePortParameters)
  )

  lazy val module = new LazyModuleImp(this) {
    val io = new Bundle {
      val in = node.bundleIn
    }
  }
}

Connect master to slave

class Slave()(implicit p: Parameters) extends LazyModule
{
  val ahbSlaveParameters = AHBSlaveParameters(
    address = Seq(AddressSet(0x0, 0xFFFF)),
    regionType    = RegionType.UNCACHED,
    executable    = true,
    supportsRead  = TransferSizes(1, 4),
    supportsWrite = TransferSizes(1, 4)
  )

  val ahbSlavePortParameters = AHBSlavePortParameters(
    slaves = Seq(ahbSlaveParameters),
    beatBytes = 4
  )

  val node = AHBSlaveNode(
    portParams = Seq(ahbSlavePortParameters)
  )

  lazy val module = new LazyModuleImp(this) {
    val io = new Bundle {
      val in = node.bundleIn
    }
  }
}

Top level hookup

class ChiselTopTest extends ChiselFlatSpec {

  class ChiselTopTester(c: ChiselTopWrapper) extends PeekPokeTester(c) {
    println("Hello World!")
  }

  // https://github.com/ucb-bar/rocket-chip/issues/359
  class ChiselTopWrapper(p: Parameters) extends Module {
    // must wrap Module into Module() ..
    val top = Module(LazyModule(new ChiselTop()(p)).module)
    // must wrap IO into IO(), thought it wasn't necessary with Chisel3 ..
    val io = IO(top.io.cloneType)
    io <> top.io
  }

  implicit val p: Parameters = new BlankConfig
  chisel3.iotesters.Driver(() => new ChiselTopWrapper(p)) {
    c => new ChiselTopTester(c)
  }
}

Generated verilog (abridged), woo!

module ChiselTop(
  input   clock,
  input   reset,
  input   io_ddrClock
);
  Master_master master (
    .io_out_0_hwdata(master_io_out_0_hwdata),
    .io_out_0_hrdata(master_io_out_0_hrdata) //--TODO: check naming
  );
  Slave_slave slave (
    .io_in_0_hwdata(slave_io_in_0_hwdata),
    .io_in_0_hrdata(slave_io_in_0_hrdata)
  );
  // slave to master for read data
  assign master_io_out_0_hrdata = slave_io_in_0_hrdata;
  // master to slave for write data
  assign slave_io_in_0_hwdata = master_io_out_0_hwdata;
endmodule

... there might be a bug in naming of ports for hrdata.