"Fundamentals of Digital Image and Video Processing" - A Course Review

I finally completed the "Fundamentals of Digital Image and Video Processing" course offered by Northwestern University on Coursera. Despite having a background in signal processing and experience with image/video processing, I found the course to be quite underwhelming. In truth, I skimmed through a good 30-40% of the material and quizzes.

The course material was dense and often felt like it glossed over important concepts. The instructor's style, with slides packed with information, didn't resonate with me. It seemed a better fit for those already familiar with the subject matter rather than your typical e-learner. Maybe if I were still a student and had fellow students to commiserate through the pain? 

While the quizzes were decent, the questions appropriately vague to force you to think rather than look up answers, the programming assignments were underwhelming. Look elsewhere if you're looking to learn about digital image and video processing.

Happy New Year! Welcome 2023.

I have been tremendously fortunate this year (and past several years). At work, no big moves, have landed into a routine over at Google. Not a bad thing and no complaints. For Tensor G2 announced earlier this year, my primary focus was on its camera hardware.

Post-COVID, I needed to get back into the world. I traveled to Belgium on a whim. For Fall 2022 semester, I volunteered as a Leadership Coach at SJSU UNVS 101 on behalf of Braven non-profit. My role was to facilitate weekly sessions of the Braven Accelerator course. It was a great experience, great people all around. I am thinking of returning next fall. Looking forward to further community engagement next year.

Back online and VSCode!

A kind individual reached out to inform me my blog went down. To myself, I said, I have a blog? 😃 Quick fix to DNS records and we're back online.

I've jumped on the VS Code bandwagon. I was skeptical at first. The desktop front end is sufficiently lightweight and responsive. Working with web alternative VS Code Server is convenient and provides the same (or virtually the same?) experience as the desktop version. So far, I've been impressed with available extensions and the growing community. On the daily, I use Python extension for VS Code, Prettier, and VS Code Vim.

The cherry on top and what sealed the deal for me are the json-based, revision control friendly configuration files. I shared mine in my chezmoi dotfiles repo.

One problem I faced was sharing my config files between VS Code and VS Code Server. Assuming the default location of ${HOME}/.config/Code/User/settings.json in Linux. You can start VS Code Server with the following to pick up the shared settings:

code-server --user-data-dir ${HOME}/.config/Code

What's Been Up?

I can finally broadcast what I've been working on since I shut down my company at the end of 2018. I have been at Google working on:

https://blog.google/products/pixel/introducing-google-tensor/

For Tensor GS101, I primarily worked behind the scenes as lead, then manager, of a design verification CAD team. I later switched to a design verification lead role for YouTube's Argos ASIC, specifically on the video codec IP written using C++ and synthesized using HLS. Following some reshuffling, I continue doing the same, but for something different, still 100% confidential. Hope to talk about it soon!

Technical paper review #1

Marvell ThunderX3: Next Generation Arm-based Server Processor

https://ieeexplore.ieee.org/document/9340376

Authors describe implementation of ThunderX3 CPU. Here are the key points that stood out for me.

ThunderX2 was derived from an earlier MIPS based architecture (which I was privileged to work on for a short while). ARM instructions that did not directly map to MIPS were expanded into micro-ops. A 6% gain in SPECint was achieved by reducing micro-op expansion.

Details on arbitration policies for hardware threads inside the OOO pipeline are presented. Arbitration decisions are made, for example, based on number of micro-ops in flight further down the pipe or the most micro-ops to retire.

The paper described the rational for choosing a ring topology over mesh: memory bandwidth not increasing dramatically over next generation.

Goodbye Company

It with a fair amount of sadness to announce that, last month, I shut down my company: P&L Systems, Inc.

These past 16 months have been quite the experience.

I recognize the importance of acknowledging (and honoring) the feelings that came up throughout the adventure. To shut down any company is a loss, and with loss comes grief. I cannot precisely identify with all of the five stages of grief (denial, anger, bargaining, depression, acceptance).

Denial and anger do not resonate at the moment, but bargaining, however.  "What if I did ..." or "If only this happened ..." statements are at the forefront of my consciousness. Relating the loss of a company to the loss of a loved one is sobering, quoting: "We want life returned to what it was; we want our loved one restored. We want to go back in time, find the tumor sooner, recognize the illness more quickly, recognize the illness more quickly, stop the accident from happening... if only, if only." The author of the quote goes on to explain that the "if onlys" cause us to find fault in ourselves and what we "think" we could have done differently. Depression set in, somewhat unconsciously, during the final weeks of the company. It showed up as a more-relaxed and scattered attitude to my daily routine. Finally,  with acceptance, I accept that this company has been my greatest failure so far. I do not accept that this was my last.

I am so incredibly grateful for the individuals that helped make the company possible. Money was lost, and that is the painful truth. I do not regret the financial consequence. I could have purchased a Telsa; instead, I invested in myself. If continuing to drive a 10-year-old vehicle is the consequence of failure, so be it, I will retake that consequence over and over again.

I cannot imagine that it will be possible to list all the lessons learned. The two lessons that most resonate with me are (1) that I underestimated the effort required to adequately cover non-technical responsibilities (i.e., networking, sales, marketing, business development, etc.) and (2) that I underestimated the complexity of building a non-trivial machine learning system (i.e., choosing the correct features, massaging input data, effectively using the results of trained models, etc.). More specifically, I should have prototyped the core technology (hard) before building the supporting infrastructure (should have been easy).

Luck (or, Karma?). That's the only way I can describe what's next for me. I am starting a new role at Google beginning January 7th. I am so incredibly humbled and grateful. Much of the specifics of the position remain unknown, but I am confident that it is a fit (short- to medium-term, at least). I plan nothing short of making an impact at the company.

Three small projects

I recently released three small projects on my personal github:

• MAME ROM filter tool
• Command line interface for edaplayground.com 
• UVM verification component and testbench generator 

Kudos to Edaphic Studio

I have previously talked about my use of IntelliJ as my primary development environment for Scala, C++ (CLion), JavaScript (Webstorm), and Python (PyCharm). I cannot recommend it enough. As luck would have it, a company is developing an IntelliJ plugin for Verilog and SystemVerilog. The plugin is in beta phase but remains exceptionally usable. The author is responsive in addressing my bug reports. Most refreshing is the tools' transparent pricing structure. No EDA sales representatives to deal with!

In summary, I wholeheartedly recommend hardware developers check out: https://www.edaphic.studio/.

Register a bundle in Chisel3

A quick update to a previous post that I made: http://blog.edmondcote.com/2017/03/register-bundle-in-chisel.html.  Here is how I now create a register (sequential storage element) of a bundle in Chisel3.  The benefit is the elimination of the previously-necessary wire declaration.

val pageReg: Page = RegInit({
  val page = Wire(new Page())
  page.initialize()
  page
})

Cramming and (Re-)Learning

I remember "ignoring" my statistics class in undergrad.  What I mean by that is I never attached real importance to the subject matter.  I did enough work to get through midterm and final exams, but not much else.  The result is that none of the content ever stuck.  I felt lucky that I was able to cram sufficiently and pull off B+ or A- grades.  In retrospect, I recognize that I could have done a much better job in absorbing the material.  Not only in statistics but in other subjects as well...

Back to re-learning.

Fix journalling issue older generation SSD performance in Linux

I found a solution to an issue that has been hampering my development efficiency for months.  The symptom of the problem was any large file I/O operations on my secondary SSD stalled my Linux system.  The drive only contains applications and it is not a working volume.

My initial guess was a failing SSD.   SMART values did not indicate as such.  The drive was only a few years old and was from some obscure brand (PNY).

IO profiling indicated that process [jdb2/sdd1-8] had high IO write wait times while the system was "stalled".  I reproduced the behavior by writing a large random file to the disk.

Fast forward a few hours of research and unsuccessful attempts at modifying filesystem settings.  The solution was to disable journaling for this "previous generation" SSD drive.  It is work noting that the problem does not occur on my "newer generation" Samsung 840 root drive.

Lesson learned?  It would have been nice to "call IT" and have them fix it.  Not much else.  I guess that learned about IO system profiling and file system journalling.  It's time to get back to real work.

# monitor journalling process
sudo iotop -obtqqq | grep "jbd2"

# write large random file
head -c 10G < /dev/urandom > tmp

# disable journaling, need to unmount first
sudo tune2fs -O ^has_journal /dev/sdd1

# drive information
=== START OF INFORMATION SECTION ===
Device Model: SSD2SC120G3LA726B087-327
Firmware Version: 560ABBF0

https://www.harddrivebenchmark.net/hdd.php?hdd=SSD2SC120G3LA726B087-327

Another Scala Cheatsheet

I recently wrote up a "cheat sheet" for Scala.

You can download a .pdf copy of the file here.

My office is coming together nicely thanks to "decorations" as such.

References

https://alvinalexander.com/downloads/scala/Scala-Cheat-Sheet-devdaily.pdf (75% “copied” from his material)
https://alvinalexander.com/scala/how-to-define-use-partial-functions-in-scala-syntax-examples
http://blog.originate.com/blog/2014/06/15/idiomatic-scala-your-options-do-not-match/ (see: The Ultimate Scala Option Cheat Sheet)

Trying to gain traction

Few updates this time. The company is still in "stealth" mode and trying to gain market traction. The target market is quite small, and that is OK. The original business plan calls for developing partnerships with other businesses.  Unfortunately, one potential opportunity is on hold for reasons outside of my control. I have not spent sufficient time on marketing over the past 4-5 months.  This does not mean going out of "stealth" mode.  I expect it will happen before 2019.  Personal cash flow is not yet an issue.  However, I am slowly beginning to look for suitable (i.e., interesting) short-term contracting opportunities.  The timing of the search is arguably premature but I do not have a good sense of what the market for shorter-term contract work is like.  Therefore, some caution is necessary.

I have been updating my gist repository on GitHub several times per week. Check it out here.

Using IntelliJ as RocketChip IDE

• Checkout the code according to instructions here.

$ git clone https://github.com/ucb-bar/rocket-chip.git

$ cd rocket-chip

$ git submodule update --init

$ export ROCKET_CHIP=`realpath ./rocket-chip`

• Build and publish FIRRTL, as usual:

$ cd $ROCKET_CHIP/firrtl

$ sbt clean publishLocal

• Verify that Rocket Chip can be compiled outside of IDE:

$ cd $ROCKET_CHIP

$ sbt clean publishLocal

• Wipe existing IDE configuration .idea, if necessary:

$ rm -rf $ROCKET_CHIP/.idea

• Import project into IntelliJ

Navigate to build.sbt, select default settings.

• (Optional) Attempt to compile using sbt, within IntelliJ:

• Rebuild project using IntelliJ

• Now for the secret sauce, write your generator object:

Start with running freechips.rocketchip.groundtest.Generator.

It will fail.  That is good.

Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: Usage: sbt> run TargetDir TopModuleProjectName TopModuleName ConfigProjectName ConfigNameString

at scala.Predef$.require(Predef.scala:224)

at freechips.rocketchip.util.GeneratorApp$class.names(GeneratorUtils.scala:76)

at freechips.rocketchip.groundtest.Generator$.names$lzycompute(Generator.scala:7)

at freechips.rocketchip.groundtest.Generator$.names(Generator.scala:7)

at freechips.rocketchip.groundtest.Generator$.delayedEndpoint$freechips$rocketchip$groundtest$Generator$1(Generator.scala:8)

at freechips.rocketchip.groundtest.Generator$delayedInit$body.apply(Generator.scala:7)

Fixing this is left as an exercise to the reader, but I will provide this simple (and, oh so powerful, tip).

Right click on 'GeneratorApp' from line 'object Generator extends App' and click 'Go To'.  This will bring you the definition of the GeneratorApp trait.

So, why not add a breakpoint and debug yourself?  The process is outside of the scope of this post.

• Tips for advanced users:

I typically do not do work inside the rocket-chip repo.  Similar to the firrtl publishLocal step, I do a publishLocal of rocketchip and import the jar into my project.  My generator function is highly customized.

See the following for more information:

https://github.com/freechipsproject/rocket-chip/pull/1319

Super! Busy!

I am always working.  It is a good thing and a bad thing.

It took me a long time, but I officially made the Red Hat to Ubuntu switch.  I have been a Red Hat/Fedora user since version 6 (nope, not RHEL 6).  That is nearly twenty years.  Why?  It is nothing against Red Hat.  Ubuntu simply has a stronger community.

Publishing my gists (i.e., ramp up notes)

Detailed notes are important when ramping on new technologies.  I recently decided to publish these notes at: https://gist.github.com/edcote

Address Masking for RAM Devices

I found the problem of generating the correct address for a memory operation confusing and decided that it would be of value to me (and others) to take notes.

Assume we have an implementation of a slave device on a TileLink2 bus. It can receive either read (Get) or write (PutFullData) messages on channel A and correctly return the response on channel D.  More information about this in the spec.

The following covers the implementation of the module, specifically on the mask generation, nothing more.

Assumptions. The slave device covers address range 0x2000000, size 0x1FFF (or 8KB).  The system bus uses 8 bytes per beat (matching the 64 bit bus).  The memory width matches the bus width.  Number of memory words is 1024 and address width is 10 bits.

val address = in.a.bits.address // receive the full 64-bit physical address

val size = 0x1FFF // 8KB size, get using addressSet.mask

val size = in.a.size  // log2(bytes) of total amount of the data requested, e.g., 8B cache line == 1

Problem.  To compute the word-aligned address, you need a mask.  Here is an example worked out by hand.

2000_1000 ^ (0x2000_1000 & ~mask-1) = 0x1000 (byte aligned) // where mask = 0x2000

1000 / 8 = 0x200 (8B word aligned)

addr[9:0] = 0x200

That was the easy part.  I got confused by the operation of the circuit generator.  Isn't the following great ;)

val a_address   = Cat((mask zip (in.haddr >> log2Ceil(beatBytes)).toBools).filter(_._1).map(_._2).reverse)

This is definitely one the least appealing parts of functional programming.  It would take me a few hours to reverse engineer this.  How this should work is any non-trivial function should be placed in its own container and unit tested.  The unit tests are commented and serve as a description of the functional algorithm.  But, I digress ...

There is a helper object in the util package.  Moral of the story.  Use it.

// This gets used everywhere, so make the smallest circuit possible ...

// Given an address and size, create a mask of beatBytes size

// eg: (0x3, 0, 4) => 0001, (0x3, 1, 4) => 0011, (0x3, 2, 4) => 1111

// groupBy applies an interleaved OR reduction; groupBy=2 take 0010 => 01

object MaskGen {

  def apply(addr_lo: UInt, lgSize: UInt, beatBytes: Int, groupBy: Int = 1): UInt = {

[..]

Mid-January Update

November was mostly about writing a grant proposal for DARPA.  It has been 110% technical work since then.  Beginning in February, I will shift to writing pretty PowerPoint slides for the company.   I should have a minimum viable prototype by then.

I worked on an experiment to build a continuous integration environment for the software assets of the RISC-V Foundation; learning Groovy and Gradle the process.  The prototype uses Jenkins2 for which I have been pleasantly surprised.  My previous experience was Jenkins 1.x.  I published source code for the prototype here.  I plan to volunteer some time to help coordinate some this activity.  It is a small <5%-of-week commitment.

I began to move parts of my internal wiki to GitHub gists.  I plan to add more over time.

Finally, I am accumulating notes on how to bootstrap RocketChip.  You can find the notes here. 

Also, compared to six months ago, I am beginning to feel very comfortable writing functional code using Scala.  I struggle a bit here and there (i.e., when recursion comes into play), but overall it is a positive experience.  If I had time, I would love to take an advanced class.  It would be interesting to re-learn data structures, algorithms through a functional lense.

Debugging Spike with CLion/CMake

Another quick entry.  I released a template project on GitHub that allows you to build, run, and debug spike, the RISC-V ISA simulation, using CLion.

https://github.com/edcote/spike-cmake

Clone, import the template, and enjoy!

CLion Kool-Aid

Doesn't it feel good to purchase a product/service from a company and be damn happy?
I have been resisting purchasing this product for a few years, heck; I even tried to pirate it (for personal use) at some point.

CLion!

https://www.jetbrains.com/clion/

That's all.  CLion is an incredible tool for C++ development, and you too should buy it.  Integration is seamless with Linux/Unix development tools: git, gdb, cmake, clang, etc. Most importantly, use of the product doesn't interfere with others that choose more-traditional development styles.