Bending ePaper To Our Will with Custom LUTs

Intro

At TRMNL, we're always trying to improve your user experience.

Last year our first firmware release relied on software provided by our display vendor to update images on our ePaper screens. This vendor, like many vendors, provided just a single way to use the display known in the industry a "full update."

Full updates clear the display to all black and all white in multiple passes before finally showing the image you sent it. This lasts for several seconds and is unpleasant to watch. I'll explain why it does this later in this article.

Before joining the TRMNL team I sent a message to Ryan (TRMNL's founder) that I could potentially help improve the display code and other areas of the software. Part of the reason I was confident I could help is because the display is controlled by software and software can be rewritten.

Investigate what's possible instead of accepting what's shown. Assume there's a better way to do everything.

Modern life floods us with a constant flow of information; too much for anyone to fully process. In order to try to make sense of it we create an internal list of information dotted with assumptions based on personal experiences and what we're shown.

When it comes to modern technology, most people will assume a product can only operate in the way they saw it operate. This is often the case and is a rational way to handle the endless firehose of info we're presented. I personally prefer to know how things work and sometimes spend too much time investigating a narrow topic until I'm satisfied I fully understand it or have fully tested its limitations.

In the case of ePaper displays, I've spent a few years getting to know the technology and have a pretty good understanding of how it works and its limits. Before we start, let's go over some of the terminology:

• SPI - Serial Peripheral Interface; a relatively low speed interface usually consisting of one data bit.
• EPD - Electrophoretic Display; the technical term for ePaper displays.
• Eink vs ePaper - Eink is a company name with a registered trademark; ePaper is the industry name for elecrophoretic displays.
• LUT - Look up table

How ePaper Works

The black and white displays that we've seen in e-readers and other products are based on an array of small spheres containing a clear oil and small particles with black and white pigment, each with a positive or negative electrostatic charge.

On the visible surface of the display is a transparent electrode and on the opposite side is a semi-transparent electrode. A voltage is applied across the two electrodes of each pixel which creates an electric field and causes the charged particles to move (the black and white particles will move in opposite directions). Depending on the polarity of the electric field applied, the black or white particles are pushed up or down. When the electric field is removed, the particles remain in their last position; this allows for images to stay on the display without the need for any power.

The particles take a finite amount of time to move from one color state to the other; this time can increase at lower temperatures because the oil becomes more viscous. The movement of the pixels is surprisingly fast - on the order of microseconds. Changing the contents of the whole display can take much longer (hundreds of milliseconds for SPI screens) due to the nature of the low power controller and the update process which steps through the rows one at a time.

If the pixels were connected with individual transistors instead of a matrix arrangement, it would probably allow for much faster displays.

SPI ePaper Controllers

There are two main vendors of specialized microcontrollers which control the arrays of pixels which make up SPI ePaper displays: Solomon Systech and UltraChip.

These controllers were designed to be extremely power efficient and quick (conflicting goals for computer equipment). They are so efficient with power that they can perform thousands of updates from a single CR2032 coin cell.

Each company's controllers do the same job, but slightly differently. They both use single byte commands to configure the update process, but the commands and parameters are all different. The two controller command and parameter differences can be abstracted away into a single API (like my bb_epaper library), but it still presents some challenges for the end users.

One of the worst problems is that each chip vendor has made changes to their commands and parameters over the years and there is no easy way to know which version is in the panel you're using. ePaper displays are typically configured as write-only devices, so even if the controller could identify itself, the SDO (serial data out) signal from the controller isn't connected on most projects/products.

This is where the danger lies - using the wrong parameters or commands can damage or destroy the panel's power circuit.

SPI ePaper Update Modes

The specialized microcontrollers have built-in RAM which holds the contents of the display. The black/white ePaper displays have two 'planes' of 1-bit memory. Each 1-bit plane can define the image displayed on the screen, but the designers of the controller chips were clever to create much more functionality by using two memory planes.

The first plane is capable of doing 'full/slow' updates and fast updates of 1-bit images. The second plane enables two special use cases: In 1-bit black/white mode, one memory plane holds the new pixels and one holds the old. A differential update will compare the two and update only pixels which changed color. It then (optionally) copies the new memory into the old in preparation for the next update.

You've seen differential updates in action - they're the ones that don't flicker and the pixels just smoothly transition from the old image to the new. The other special type of update is 4-gray mode. In this mode, the two memory planes are used together as bit 0 and 1 of each pixel to display 4 possible gray levels. These four possible update modes (full, fast, differential, 4-gray) are not built in to the software of every ePaper controller. The original TRMNL ePaper display didn't offer fast mode nor the 4-gray mode we have today.

Defining an ePaper update mode

The ePaper controller has a fixed 'program' in it which performs updates by using info from a LUT (look up table). It loops through each row of the display, one at a time and modifies the pixels based on the contents of memory and the LUT contents.

The LUT contains timing information and sets of 2-bit 'push' sequences which instruct the hardware like so:

00 = do nothing
01 = push towards black
10 = push towards white
11 = undefined

There are four sets of push sequences depending on the contents of the two memory planes. For UltraChip controllers, it's a bit easier to see how the LUT behaves because it explicitly breaks the update into separate LUTs:

1. WW (1->1 or white to white),
2. WB (1->0 or white to black),
3. BW (0->1) and
4. BB

These 4 tables tell the controller how to push each pixel which is represented by one bit from each plane (old->new). On Solomon controllers, the 4 LUT options are all defined in a single table. The push sequences can be repeated any number of times and the time duration of each push can be controlled too. ePaper controllers come with built-in LUTs and allow you to define a custom one.

The one advantage that the internal LUTs have over custom ones is that there are multiple versions of each LUT, selected depending on the ambient temperature. If the internal temperature sensor is enabled, it will automatically pick the right LUT. If you provide your own LUT you need to make it appropriate for the current ambient temperature. At temperatures around 20C (+/- 10C), a single LUT can work well. It's primarily at very low temperatures that more/longer pushes are needed.

The Do's and Don't's of LUTs

There is a lot of freedom in how you craft a custom LUT. The built-in LUTs tend to be very generous with your time to make sure that all of the pixels look their best; in other words, they're slow.

Let's say you want to do a super fast non-flickering update that doesn't depend on knowing the previous color of each pixel. You can just push black pixels towards black and white pixels towards white without caring if they were already that color or need to change. This will work... at first.

Over time, the pixels can actually get a little bit stuck after being pushed repeatedly towards the same color. It's normally not a permanent problem, but can require a bunch of black+white full erasures to get them to behave normally again. In ePaper terminology this is called the 'balance of charge'. It means that if you push a pixel N times towards black, you should push it N times towards white to counteract that set of pushes.

Other factors affect the success/stability of updates - a LUT which doesn't do a full black/white wipe and doesn't over push pixels may see some 'ghosting' after a series of updates. This term means that you can see some black pixels which aren't fully black and vice-versa. This is why it is recommended to do a 'slow/full' update after every 10 fast updates to make sure all pixels are reset to fully black and white.

Newer EPD controllers usually include 3 internal LUTs - full (~ 3 seconds), fast (~ 1.5 seconds) and partial (~ 0.5 seconds). These have (in order) - multiple black/white flashes, a single black/white flash, no black/white flashing. It's up to you to use these different update methods prudently.

What's in a LUT?

If you're more interested in the big picture and less in the fine details, you can skip to the next section. Below is a screenshot from the UC8179 datasheet.

Here I'm using an UltraChip LUT as an example because it's easier to comprehend compared to Solomon's various LUT versions. On UC81xx EPD controllers, there is a separate LUT for each 'color'. In this case, the LUT defined by command 0x21 is for White-to-White. This means pixels represented by a value of 1 in both the old and new bit planes (aka 1->1 or White->White).

Each row in the table is one byte, each group is 6 bytes there are 7 groups (42 bytes total). Think of each 6-byte group as a 'phase' of the update and you can define up to 7 phases. In each update phase you define up to 4 pushes, each with a variable number of 'frames' (a frame is a pass over every row of the display).

For my custom LUTs, I usually only need to define a single 6-byte phase because I can accomplish what I need in four pushes. The frame timing is controlled by register 0x30 as shown below:

The faster the frame time, the less each pixel is pushed along its path by each frame. The default frame time for this controller (50Hz) means that for a display with 480 rows, each row is active for <= 41.6 microseconds per frame. Here's an example of one group (6 bytes) of W->W for a fast (1.0 seconds, single flash to black) update:

0x60, 0x18, 0x19, 0x01, 0x00, 0x01

0x60 = 01 (black) 10 (white) 00 (neutral) 00 (neutral)

Black is pushed for 24 (0x18) frames, followed by white for 25 (0x19) frames, followed by 1 frame of neutral. It is run one time (byte 5 == 0x01). 50 frames at 50Hz = 1 second. The other 6 groups of the LUT are set to 0 (inactive).

Creating grays on a black/white EPD

ePaper displays are designed to display black and white pixels; gray is really an undefined/indeterminate position of the colored particles.

As I described in the previous section, you can update the pixels without knowing their previous state by simply pushing towards black and white sufficiently to get them to the 'end' positions of their movement.

If, however, you know their current state, you can push them a little less than a full push and they will end up somewhere between black and white. So, if you do a 'clean' phase by pushing them all towards black and then all towards white, you know the starting position of every pixel.

The next steps in your LUT can push the pixels towards black by a varying amount depending on the gray level you want to achieve. The more gray levels you want to create, the more difficult it becomes. The challenge is that these are physical particles which are affected by inertia, velocity and the variable viscosity of the oil. The other challenge is that these controller chips only have 2-bits of memory per pixel (4 possible states).

If you want to create more than 4 possible gray states, you would need to use a second LUT and re-load memory with new information to adjust the pixels into 4 possible new states from the last update.

This is why SPI ePaper panels only offer a maximum of 4 gray levels.

Parallel Eink panels (e.g. e-book readers) controlled by more powerful systems usually offer more than 4 gray levels, but rarely more than 16. It's very difficult to get more than 16 unique gray levels that are somewhat linear in intensity due to the nature of how the ePaper pixels work. There are also some unwritten rules about how the display behaves.

One of the things I discovered experimentally is that active pixels (being pushed towards black or white) affect neighboring pixels that are idle during that update pass. This results in visible 'noise' on the boundaries of pixels of different shades of gray. It makes sense intuitively because if there's a strong electric field generated for a pixel, the neighboring pixels will feel some influence from that same field. The way around this is to have all pixels active during every update pass (no neutral steps in the middle of an update).

Fast Grays?

With knowledge of the physics of ePaper, along with the SPI controllers and their LUTs, we can now craft a fast grayscale mode.

For my first attempts at creating 4-gray modes, I followed the traditional (vendor's) path and started the update with a couple of cycles of black and white to clear the pixels to a known state, then had the pixels do some 'dancing' between white and black to arrive at a nice looking gray value. This update program lasted around 2.5 seconds and produced good looking results.

I recently came up with a simpler idea for 4 gray levels - we can arrive at the desired intensities directly without the need to clear the display first. This idea bends the 'balance of charge' rule a bit, but won't hurt the display. To quickly get 4 gray levels, I set up a LUT to do the following:

black: push fully towards black
dark gray: push fully towards black and slightly towards white
light gray: push fully towards white and slightly towards black
white: push fully towards white

This update pattern only takes about 600 milliseconds and creates the least visual disturbance for the user. You can see a slight flick of color change for light and dark gray pixels while black and white transition smoothly. A slightly slower variation of this would introduce a single black+white wipe before doing the final steps.

Conclusion

As you can see in the image below, the grays produced by the fast LUT look reasonable, but there are no guarantees that they are perfectly linear from white to black. This will be true for LUTs provided by the panel vendors and custom ones that you create.

Custom LUTs can be applied to pretty much any SPI ePaper panel, just beware of the many variations of controller parameters and LUT table definitions.