path: root/docs/umich/f24_473.md

# Fall 2024 Course Review: EECS 473

2024-12-28

Course Title: Advanced Embedded Systems

Rating: 4/5

## Instructors (Mark Brehob, Matthew Smith, James Carl)

I commented on Mark in the first semester, regarding [EECS
370](f23_wrapup.md#instructor-mark-brehob). He hasn't changed. I have.
I've been working on my social aptitude and interacting more on lectures.
I also drop by his office hour a lot more frequently than I used to, often
to discuss our project.

Matt worked with Robert Dick on
[373](w24_373.md#instructors-robert-dick-matthew-smith) last semester.
I had less interaction with him than in 373. I spent a lot more time with
our GSIs though.

James Carl was recently hired to replace Matt once he retires. He has some
pretty good debugging ideas.

## Course topics

These are topics of significance in my opinion:

- Interface
    - The Arduino library is an interface
- RTOS (real-time operating system)
    - Task scheduling
    - Semaphores
- Copyright and copyleft
    - GPL
    - Fair use
- Linux
    - "Everything is a file"
    - Device drivers
    - Kernel modules
- PCB
    - Making of PCB
    - Power integrity
    - Bypass capacitors and frequencies they can handle
- Batteries
    - Choosing a chemistry
    - Estimating battery life
- Voltage regulator
    - LDO (linear regulator)
    - Switching regulator
- DSP
    - "Specialized computing"
    - Fixed point arithmetic
    - Common DSP algos
- Wireless
    - Hamming code
    - Keying (ASK, FSK, etc)
    - Shannon's limit
    - Antenna range

There are five labs in total, but I was exempt from the last one because
I knew how to design PCBs already. The labs are super high workload and
never once did my teammate and I finish them within the three hours of lab
time. We always worked overtime, sometimes on Saturday.

I'm relieved that we aren't using STM32CubeIDE or dealing with their
horrid code style anymore.

## Project

When I enrolled in the course, I thought the goal of the project was to
create the best PCB. It was not. Instead, what we were supposed to create
is a product — with a use case and a market. It's OK if the PCB is full of
bodges. Just make it work.

Teams of 4-5. Budget is $200 / person. I'm in a team of 4. Our project is
the Live Caption Badge. Taken from project report:

> What we made was the Live Caption Badge. At the CoE Design Expo, we
> demonstrated the device by inviting visitors to wear the badge and talk
> with us. With the press of a button, the visitor can hear our voice, and
> we can hear theirs. We also showed the visitor that their speech was
> transcribed live on the screen.

> The goal of our project is to improve communication in noisy
> environments, such as convention halls, for hearing-impaired individuals
> and non-native speakers. Our solution is a live caption badge that
> records and uploads speech to a server running a transcription API. The
> badge then displays the text on an e-paper screen. Badges can form
> a talkgroup, where users can hear each other’s speech.

![An e-paper display in a 3D-printed box. On the screen is the UMich logo,
Frederick Yin, he/him, they/them, Speaker.](img/f24_wrapup/lcb.jpg)

I will now expand on the subsystems, discuss the work I was involved in,
and the manmade horrors beyond comprehension.

### E-Paper

When I brought this idea to Mark, he was skeptical, especially about the
epaper. He thought epaper was slow, but he didn't know the model we picked
([Waveshare 7.5-inch](https://www.waveshare.com/7.5inch-e-paper-hat.htm))
supports partial refreshes, which is a feature where a rectangular area
gets refreshed in only 0.5 seconds. He still was not fully convinced, but
we went with the epaper anyway because (1) it's readable from a wide angle
and (2) it's cool and we have the budget.

Waveshare published driver code, which worked as long as you're super
careful with it. For example, there's a function for partial refreshes,
which takes an `xStart` variable among others. What's implied, and kinda
obvious in hindsight, is that `xStart` must be a multiple of 8, because
each byte is 8 pixels. However, there was nothing in the documentation.
When I passed an illegal value, it didn't even warn me. It's not a great
software interface.

I also spotted a data underflow error. They did something like `size_t
s = (x % 256) - 1`, which might underflow to `(size_t)(-1)` if `x % 256 ==
0`. It should have been `(x - 1) % 256`.

The coding style is so inconsistent that I didn't even bother fixing it.

When I was designing the PCB (I started late and missed the deadline,
whoops) I realized there was no time to design the driver circuit for the
epaper. So I asked Mark if we could just piggy back the breakout board on
our PCB and he reluctantly said OK.

So:

![PCB with a notch on the bottom](img/f24_wrapup/lcbpcbrev1.jpg)

I actually had three plans to make it work. You'll notice a row of 2.54 mm
headers and an unmounted clikmate footprint underneath. One of them needs
to be wired to the breakout board. It can be jumper wires plugged to the
2.54, or stripped wires soldered to the 2.54, or specialized cables into
the clikmate.

The breakout board has a clikmate connector, and is shipped along with
a clikmate-to-2.54 mm cable. These work directly with this board above.
I looked for clikmate-to-clikmate cables online in hope for a tidier
wiring, but they don't sell 9-circuit cables anywhere. It goes from 1 to
8, then directly to 10. Where'd Waveshare even get these?? :floofwhat:

In November I was feeling ambitious, so I decided for another spin of the
PCB, this time with the epaper circuit in-house. I left out the 2.54 mm
headers, assuming the FPC connector would work. Problem? It didn't work.
Reason? No fucking idea.

I tried probing continuity, but when the ribbon cable is fully inserted
there wasn't any exposed metal. It was days until design expo, so I quit
the struggle (correct decision). I just stripped the wires and soldered
them directly to the ESP32.

![A PCB with wires running down to a breakout
board.](img/f24_wrapup/lcbpcbrev2.jpg)

These wires annoy me. They haunt me even after the semester ended and
I got my final grades. I must eradicate their existence from the surface
of the Earth.

One day I was at MESH, and I looked up my LCSC shopping log. And there
stood what I had sought:

![Screenshot of two FPC connectors, both are Bottom
Contact](img/f24_wrapup/fpc.png)

They're supposed to be top contacts.

Shaking my head, I ordered a bunch of top contact FPC connectors from
Mouser. They worked.

![Mosfet (my fursona) standing emotionless in front of
a datapath](../emote/mosfet/mosfet_lc2k.png)

### Audio

When the project began I was fearless. GSIs and James had proposed that we
use an I2S microphone module instead, but I, being arrogant, insisted that
we use an off-the-shelf customer service headset and plug it into the
board via a 3.5 mm jack. Design-wise, I made the right choice because
a 3.5 mm is highly compatible. However, the connector caused me
a ridiculous amount of pain.

![A confusing mechanial drawing of a 3.5 mm audio
connector](img/f24_wrapup/aux.png)

This is a TRRS (tip-ring-ring-sleeve) audio connector. The datasheet just
made no sense to me. I never touched an audio connector in the unsoldered
form, so I don't know which ring goes where. Specifically, I assumed the
pad labeled 1 on the top right was the tip, because it was the farthest
away from the hole. I found a similar footprint in the KiCad library,
which I modified so that pin 1 is the tip.

To get the audio, we used an ADC chip (ES7210), which was picking up
a really weak signal amongst the noise. After ruling out soldering issues,
for days I suspected that the ADC has an inadequate input impedance. I did
calculations I learned in [311](w24_311.md). I even considered a buffer opamp.

I brought the question to Mark. He wasn't too fond of what I proposed, and
instead propose that I try different resistor values on the drain of the
electret mic. I went back to the lab to try it out.

Probing the connector with a multimeter in continuity mode wasn't helpful,
because I didn't have an aux cable with all four contacts. Oddly, probing
two of the pads sometimes make the multimeter beep, which I didn't think
too much until I saw something cursed.

I reenacted the PCB circuit upon the headset jack, bypassing the
connector. __It worked.__

![Alligator clips and probes from signal generator and oscilloscope
clipped to a 3.5 mm jack and a resistor just rolled onto the
jack](img/f24_wrapup/rig.jpg)

▲ Valid, totally not shoddy testing rig

Conslusion? The KiCad footprint was right. I was wrong. Pin 1 was the
sleeve. :floofangry:

Because of the wrong pinout, what it ended up sampling was a headphone
coil, which _was_ picking up sound but terribly. If there wasn't audio at
all, I might have come to the conclusion sooner. The beeping from the
multimeter was because the headphone coils were 32 ohms, lower than the
threshold of what makes a "short". If I had probed in ohm mode, I could
have known sooner.

The discovery both relieved me and disappointed me. I knew I wasn't smart
enough to make mistakes involving input impedance. I felt like a clown.

![Mosfet (my fursona) wearing clown makeup and a shirt that says
"Clowncy"](../emote/mosfet/mosfet_clown.png)

Hot fix: I unsoldered it and bodged some wires and lived with it until Rev
2.

![Four wires soldered between footprint and connector pads, leaving the
audio connector hanging in the air](img/f24_wrapup/auxhack.jpg)

Lesson: always check pinout before proceeding with any troubleshooting
that requires a brain.

### USB-C

I put three connectors on the PCB (other than 2.54 mm headers). __Every
single one of them__ had some sort of issue.

I talked about the audio connector. I talked about the FPC connector.
I also made a mistake with the USB connector. You see, in an attempt to be
"hip", I insisted on USB-C, and bought some from Mouser. The problem was,
I missed something on the datasheet again. The area where the metal casing
goes is not just a keepout — it's a physical notch in the board cutout. So
in Rev 1 we had to use an FTDI dongle connected to the UART headers via
jumper wires. Every time we had to flash firmware, we had to:

- Hold Reset and Flash buttons
- Release Reset
- Release Flash
- `idf.py build flash monitor` (aliased as `bfm`)
- Press Reset

In Rev 2, the USB worked on the board my teammate Angel soldered but not
on mine. In a fit of rage (in my defense I was tired) I ripped the
connector off, which did permanent damage to the copper traces, so we had
to keep using FTDI on my board.

Both of my friends who work in embedded agree that connectors are the
worst.

### PCB

This PCB is special to me. It is the first time I designed a PCB so large
(Rev 2 is 170×105 mm, although most of it is empty, which did leave space
for me to sneak in some artwork).

The board was my first time doing 4-layer. (Middle layers are just 3.3V
and GND.) James advised me to do so because he feared the signal lines
dragging across the PCB would pick up interference.

It's also the first time I ordered a stencil for my own design. It's also
the first time I used an ESP32 that's not on a DevKit.

The size, layers, and stencil combined made PCBs real expensive.

- Rev 1
    - PCB: $42.61
    - Stencil: $7.11
    - Shipping: $42.15
    - Discount: -$10.00
    - Tax: $4.92
    - Total: $86.79
- Rev 2
    - PCB: $43.41
    - Stencil: $7.11
    - Shipping: $33.28
    - Tax: $4.67
    - Total: $82.47

The shipping skyrocketed (basically doubled) once we ordered the stencils.
In total we spent ~$169 on PCBs, or 21% of our budget.

We spent $225.50 on stuff that go on PCBs, 28% of our budget.

The DAC and ADC audio chips are QFNs with 0.4 mm pitch, the finest I've
ever soldered.

The three buttons are surprisingly tactile. They're really snappy. Worth
every penny.

I left a bunch of easter eggs on the PCB.

!["Hey put it back!" under the ESP32](img/f24_wrapup/hey-put-it-back.png)

▲ Inspired by the [Eurofurence 28
badge](https://github.com/eurofurence/ef28-badge).

![":3" under Q1](img/f24_wrapup/colon-three.png)

▲ Q1 is an N-channel MOSFET.

!["pain" under the audio connector](img/f24_wrapup/pain.png)

▲ This is Rev 2.

!["Do not detonate" next to an electrolytic
capacitor](img/f24_wrapup/do-not-detonate.png)

▲ It's 68 µF.

![My fursona depicted as the Yippee creature on the
back](img/f24_wrapup/pcbback.png)

▲ Rev 1 was just the original Yippee creature, but Rev 2 has cat ears and
a tail

### 3D printing

My teammate Kyle did the 3D modeling and printing all by himself. He has
a Bamboo at home. He printed two versions for the two PCB revisions, two
pieces each. They're actually really well-made. Like, from a distance you
can't even tell they were 3D printed.

Since the FPC didn't work, there had to be a breakout board. Instead of
going on the PCB as I had intended (it didn't have enough horizontal
clearance for the big row of headers on the other side I'm not willing to
remove), it was mounted on a 3D printed piece with M3's, between the
epaper and the PCB. However, only the case designed for Rev 1 had that.
So, we used the Rev 2 PCB with the Rev 1 case, which is a couple
centimeters thicker. No one cared.

The case was black, red and white, with some serious NES vibes. Its hefty
build also reminded me of lead-acid batteries.

The 3D printed case is a huge, I repeat, HUUUGE part of our success. It
really made our project look like a product, even when it's just sitting
there. It attracted many people on the design expo, and even though it
wasn't central to the course, we got more questions about the case than
the PCB, because you can see that. We're so lucky to have Kyle.
:floofheart:

### Firmware

We had a choice between Arduino and ESP-IDF, and we were forced into the
latter because the audio library on Arduino wasn't working.

Writing an application with ESP-IDF is so. Horrible. How does anyone build
an entire application with ESP-IDF without losing their mind?

Everything is manual. Everything. The audio library on ESP-IDF is ESP-ADF,
which emulates audio pipelines, which I'm familiar with (playing with
Ardour and Pipewire and stuff). It's a neat idea and it works, but our
code is a mess. An excerpt
([source](https://github.com/fakefred/live-caption-badge/blob/main/firmware/main/audio.c#L55)):

```c
ESP_LOGI(TAG, "Create ADC i2s stream");
i2s_stream_cfg_t adc_i2s_cfg = I2S_STREAM_CFG_DEFAULT_WITH_TYLE_AND_CH(
    (i2s_port_t)0, AUDIO_SAMPLE_RATE, AUDIO_BITS, AUDIO_STREAM_READER, AUDIO_CHANNELS);
adc_i2s_cfg.type = AUDIO_STREAM_READER;
adc_i2s_cfg.out_rb_size = 64 * 1024;
adc_i2s = i2s_stream_init(&adc_i2s_cfg);
i2s_stream_set_clk(adc_i2s, AUDIO_SAMPLE_RATE, AUDIO_BITS, AUDIO_CHANNELS);
audio_pipeline_register(tx_pipeline, adc_i2s, "adc");
```

All this just to create an I2S reader. It's not even in the pipeline yet.

The rest of our code was also visually redundant and mostly unreadable,
full of hacks.

I once wanted to try writing an application with ESP-IDF. Now that I've
done that, I would never try that again unless absolutely necessary.

We were on a sprint in the last two days. Probably half of the visible
side of the software (user interaction, on-screen UI, etc.) was finished
in these two days. We sat in the lab, hour after hour, flashing and
testing firmware roughly every five minutes. It was soul-crushing. I put
on album after album, mainly twenty one pilots and Radiohead.

My teammate, Rain, suggested we add a feature where we could change the
username on the badge over HTTP, without having to reflash the badge.
I was reluctant at first, saying "it isn't a core feature", but he
insisted. Well, he was __absolutely right__.

It turns out, being handed a thing with your name on it feels different
than one without. On design expo day, multiple people agreed to have their
name put on it once we gave them the option, and some of them even took
photos!

How do we do the HTTP, you ask? Well, we exposed a couple HTTP endpoints
on the ESP32 and wrote a Python script on my laptop to issue a request
once asked to. We opened a CSV file in LibreOffice Calc called the
"Dashboard", and the Python script read from that. It was literally
written __the night before design expo__.

(The HTTP endpoints also included `/poke`, `/unpoke`, and `/reboot`.
They're hacks. Don't ask.)

### Vosk server

Another Python script running on my laptop is the server, in charge of
transcribing text and streaming audio. It's super hacky as well. It's
adapted from some ESP-IDF example code, and it was built on the super
low-level BaseHTTPRequestHandler instead of something for humans, like
Flask or FastAPI. It did allow us to commit many crimes. :floofmischief:

The text transcription was done using
[Vosk](https://alphacephei.com/vosk/), which I never heard of until Rain
brought it up as an alternative to Google Cloud, which had been planning
to use. Vosk had an advantage of being offline. It's just a pre-trained
model.

The badge-to-badge audio streamed through the server as well, which we
honestly didn't want. Ideally the audio could just go from an ESP32 to
another, without the laptop detour doubling latency. However, it was
impossible.

With our current networking stack (HTTP over Wi-Fi), if an ESP32 were to
transmit audio to another, the former has to be a client, and the latter
has to be a server. However, the ESP-IDF HTTP server __does not support
chunked reads__. Streaming audio is thus impossible without patching the
library itself, or hacks that violate the ESP-ADF abstraction.

Oh yeah and we can't really stream audio via Bluetooth (A2DP) because we
bought ESP32-S3's, which don't support classic Bluetooth, only BLE.

### Budget

![Bubble chart of what we spent. ~1/4 is
epaper.](img/f24_wrapup/budget.png)

The data:

- E-Paper: $195.33
- PCB Parts: $162.1015
- PCBs & Stencils: $94.24
- DevKits: $72.00
- Headphones: $44.93
- Misc: $73.73
- Tax: $40.18
- Shipping: $91.93

Total: $774.44 (96.8% of budget)

Notable wastes of money:

- The first ESP32 DevKit (LyraT): mic didn't work.
- The bottom-contact FPC connectors.

Rant: Our second ESP32 DevKit (Korvo-2) worked, but it didn't have
a headphone connector, only a speaker connector which wasn't 2.54 mm.
Everytime I needed to hear the output, I had to poke jumper wires into
that awkward connector hoping I didn't short anything.

This doesn't go on the budget, but about a week before design expo I was
forced to re-activate Amazon Prime (eww) to buy some lanyards and
headphones because no one else ships over Thanksgiving. I canceled it
today.

### Design expo

The instructors required "shirt and tie or equivalent", but I didn't have
a tie and a couple teammates neither. So we all just did casual. No one
fucking cared.

We set up the poster and waited for people to pay attention. We had two
badges. I wore one and offered the other to anyone interested. It was
pretty effective.

One person who goes by Michael stopped by and complimented our poster
design, saying we did a good job keeping the information density low, and
we picked a good font (Fira Code, which was the font we printed on the
epaper).

He worked in a Japanese-American company or something, and remarked that
our product had potential for communication if we had a translation
feature. He mentioned that interpreters at his company needed to translate
not only what was said, but also how it's said — for example, if
a Japanese executive yells at an American executive in Japanese, the
interpreter has to yell again in English.

I found this amusing and replied that, precisely because of this, machines
will never replace humans in communication.

Before the four-hour-long design expo, I had predicted I would either die
of fatigue, or of boredom. It wasn't too bad, and in the last hour it was
closer to the latter. I was almost always there. We consumed a bag of Cool
Ranch Doritos.

A total of four (4) furries showed up, and I totally did _not_ hand out my
new stickers in a paper bag like some kind of drug dealer.

### Report

I'm going to spare you from all the pain. Just know that it was Mark's
favorite report ("so far"). He also liked the Haiku I sneaked in the
"Lessons Learned" section. It was really validating.

The full thing:

> I am glad that we scheduled a weekly meeting early on. It enabled us to
> check in each other’s progress (or lack thereof). Another correct
> decision is to collaborate on our own branches on GitHub.
> 
> The reason why I fell behind on the PCB schedule was that we didn’t know
> what we wanted. We had never agreed on the specifics, e.g. where do the
> buttons go. Audio circuitry was also a whole new experience for me. As
> I sat in the Fishbowl 24 hours before deadline staring at datasheets,
> I felt the pressure that one wrong trace will doom our project.
> 
> As it turned out, I had not one, but five wrong traces, four of which
> being the headphone jack. Sometimes my own stupidity surprises me.
> A continuity test with an aux cord would have saved me from days of
> confusion.

> However, I must  
> Learn from my mistakes and try  
> To let go of them


And this is the end of the project. I have to stop writing it. This
article is getting close to 4K words… :floofmug:

Now, onto closing remarks.

## Arduino is good, actually: a reflection on software interfaces

I once hated Arduino. I thought it "wasn't real embedded systems", merely
a playground for those who lack the intellect to peruse the MMIO registers
on a datasheet. There was a time I avoided using it, even on Arduino
hardware itself, fearing its "overhead" would impede the efficiency of my
code. I would write firmware that, instead of a `digitalWrite`, did stuff
like `PORTC |= _BV(2)`. The [373](w24_373.md) curriculum reinforced my
affinity to bare-metal.

However, I was mistaken. Using Arduino doesn't make you less of an
embedded systems engineer. I know people who have been working as
a professional for years, and still prefer Arduino for personal projects.
This is because it's such a popular interface, so popular that basically
every tutorial was written for it, in stark contrast to ESP-IDF, which was
only ever discussed on the official ESP32 forum.

A software interface keeps ugly code to itself and presents clean function
prototypes to the user. The user just needs to be aware of a few things,
like thread safety. Most of the time the user just does stuff, and nothing
goes wrong. The interface does error checking internally to maintain
invariants.

A mistake I made when designing the epaper interface was to expose
a FreeRTOS `QueueHandle_t`. Now, my excuse was we were running out of
time. But over the winter break I've been working on Phase II of the
Badge, rewriting the firmware in C++ on PlatformIO + Arduino (without the
audio part). When writing C++, I consider FreeRTOS code "ugly", and hide
them inside class methods whenever possible.

At this point I'm on the other extreme: To avoid handling raw pointers,
I've been using the STL as much as I can, and my code looks nothing like
embedded code. It's full of `std::unique_ptr`s and `std::istream`s. It's
almost like desktop code. But hey, I've got megabytes of PSRAM and
megabytes of Flash. I can afford some tiny efficiency loss for more memory
safety. At least I don't need to `free` a pointer in every early return
path.

## Conclusion

All that said: 473 is definitely better than 373. Considering this is my
capstone, I'm even proud of it.

> Four seventy-three  
> Will never defeat me in  
> A way that matters