Implement FREETYPE_CUSTOM_GEOMETRY environment variable for unusual subpixel formats

[rennr]

This is a proof of concept

Introduction

This change to the FreeType library allows for the use of a FREETYPE_CUSTOM_GEOMETRY environment variable to globally set a custom subpixel layout when using font anti-aliasing.

The main purpose of this change is so that I could get proper subpixel anti-aliasing for my Samsung G8 OLED, which has a triangular subpixel layout. All of the current QD-OLED displays currently have the same kind of layout. None of the current anti-aliasing solutions take this unusual subpixel layout into consideration.

This issue also applies to RWBG pixel layouts, which are common on regular OLED displays.

A number of discussions on this issue have surfaced over the past year and a bit, mostly in the context of Windows. For example:

• Improved subpixel text rendering for OLED (WRGB stripe, RGB triangular) -- Unfixable by ClearType Tuner (Over 100 upvotes in Feedback Hub) microsoft/PowerToys#25595
• OLED four subpixel rendering issue snowie2000/mactype#720

What this does

The modifications to the FreeType library make it so that you can specify any arbitrary subpixel layout, and the library will use that layout when applying anti-aliasing.

You can specify this with the environment variable FREETYPE_CUSTOM_GEOMETRY, which accepts a set of offsets from the traditional RGB layout in the form of X/Y coordinates for each of the subpixels in order of R,G,B.

Namely: FREETYPE_CUSTOM_GEOMETRY=rx,ry,gx,gy,bx,by

If you set this as a global environment variable, it will apply to all applications that use FreeType. You can also set this on a per-app basis, or change it as needed.

How I use it

For my Samsung G8 OLED, the following offsets appear to provide an acceptable level of anti-aliasing without the fringing you would ordinarily see:

FREETYPE_CUSTOM_GEOMETRY=-21,-16,0,16,21,-16

I place this environment variable in /etc/environment and it applies globally after a system restart.

Limitations

1. This is a global setting, insofar as it does not differentiate between types of screens. So if you have two monitors, one of which has a normal subpixel layout, it will make that display look worse. As far as I know, there is no way to apply this change on a per-display basis.

2. This is a hack on top of FreeType. If this were to actually be implemented in the library, it would probably need to be refactored to use the FREETYPE_PROPERTIES environment variable instead of making a new one like I did here.

Building and installing

1. Clone the repository recursively to make sure you grab all the submodules
2. Run autogen.sh in the root folder
3. Run configure in the root folder. Pay attention to where your distro currently stores libfreetype.so (/usr/lib or /usr/local/lib, for example) and specify the root of that as the prefix, such as configure --prefix=/usr if your library is in /usr/lib.
4. Run make
5. Run sudo make install

If you are on Ubuntu or its derivatives, you will likely need to copy the library over to /lib/x86_64-linux-gnu as well, depending on your architecture. Be aware that overwriting the library in /lib/x86_64-linux-gnu is going to affect all applications in the running system. You will likely need to drop into a terminal outside of your desktop interface, by running sudo init 3 from your desktop (or by switching TTYs by using CTRL+ALT+F3, for example), before you copy the file over. Otherwise, your window server will probably crash before the copy completes, resulting in a borked system.

[rennr]

@mdrejhon - I've been following your posts about subpixel layout in Windows and the needed changes to ClearType. This may be of interest to you, even though it is for Linux. I've modified FreeType accept a global subpixel configuration, which should cause that configuration to be used by every application on the system.

This has effectively eliminated the fringed looks of fonts in the Linux desktop for me, and any application that uses FreeType (nearly all of the apps I use).

[mdrejhon]

This is absurdly fantastic.

I see you implement the MacType standard format for subpixel awareness.

Thank you so much for doing this. I use Linux and WSL too, though within Windows itself. However, this could incentivize me to try dual booting, just to experiment this! But I may see if I can make this run in WSL, since WSL supports FreeType if all dependences are installed. It would be quite interesting!

Someone may (eventually) want to find a way to centralize this for a customized image downscaler, like the one proposed for a virtual Windows driver, ala MolotovCherry/virtual-display-rs#35 -- although beyond scope of a text renderer, this would affect all graphics, not just text. Basically a subpixel-aware superscampled downscaler could be piped through a centralized virtual video driver architecture. Linux may have some equivalent already, but perhaps this is something you'd like to monitor too.

Pros/Cons of font renderer approach - Can stay at native resolution, higher performance
Pros/Cons of global screen approach - Requires virtualizing a higher resolution + downscaling + complex virtual desktop drivers

+1 to commit upstream to main freetype!

• Question 1: How do you adjust intensity of subpixel rendering? (e.g. blend between subpixel aware and plain antialiased)?
• Question 2: Can you post before/after photos? This will give this more traction (and turn some heads). If photographing using a smartphone; 1. Use max zoom, and 2. Move phone further away from screen until it manages to focus. Some phones do a great job, though not all. White text on black will usually be a good example of QD-OLED fringing.

[rennr]

Thanks for your thoughts, @mdrejhon!

Someone may (eventually) want to find a way to centralize this for a customized image downscaler, like the one proposed for a virtual Windows driver, ala MolotovCherry/virtual-display-rs#35 -- although beyond scope of a text renderer, this would affect all graphics, not just text. Basically a subpixel-aware superscampled downscaler could be piped through a centralized virtual video driver architecture. Linux may have some equivalent already, but perhaps this is something you'd like to monitor too.

I'll keep an eye on this, because it does sound interesting. As you observe, the downside of focusing on fonts only is that other graphics continue to suffer from these artifacts. As well, a virtual display driver would be monitor-aware, so if you have multiple different kinds of monitors, it could apply the appropriate subpixel structure to each one, rather than the global approach applied with this font hack. That said, this fonts-only solution at least appears to eliminate a majority of what (in my view) makes using an OLED/QD-OLED display less satisfying than an LCD.

With the virtual display, I'd be concerned about a few things, most importantly how well the thing would perform, in particular outside of the desktop environment (e.g., in a game). I am unfamiliar with how virtual displays work in Windows from a driver perspective, so I don't know where this thing fits in, in the pipeline. I'll have to do a bit more research.

How do you adjust intensity of subpixel rendering? (e.g. blend between subpixel aware and plain antialiased)?

FreeType already has the native ability to have arbitrary subpixel locations defined as part of its own anti-aliasing algorithm. This normally has to be applied by the application using the FreeType library. My hack just sets a default that is other than 0,0 for each subpixel. The actual way FreeType then uses this to do its rendering is not something I looked into. All I can say is that it appears to work.

Can you post before/after photos?

I no longer have the Samsung G8 QD-OLED, as I decided I needed something that wasn't an ultrawide. That said, I replaced it with an ASUS ROG PG42UQ, which has a RWBG layout and thus has a similar issue with text rendering as the QD-OLED. I took some closeup shots of the change in subpixel rendering with my FREETYPE_CUSTOM_GEOMETRY set to -24,0,0,0,0,0. I'm not sure if this is the right settings for this particular layout (the idea being that the R subpixel is moved by -24 so it's to the left of the W subpixel, but the rest of the subpixels are in their appropriate positions), but the text does appear noticeably sharper for me. I also played around with -24,0,24,0,0,0 and that looked better than the default as well.

Default 0,0:

Custom subpixel geometry -24,0,0,0,0,0:

[rennr]

I realized after posting the above that the subpixel layout is RWBG not RWGB (even though I wrote it correctly above, I didn't really consider it), so my -24,0,0,0,0,0 doesn't quite do what it should. However, it's still significantly sharper than the default.

I changed the layout to -24,0,24,0,-24,0 to swap the G and the B as well and the results are below.

There's a bit of a fringe it leaves on the stems of large letters. If I reduce it to -16,0,16,0,-16,0 then it's better. But honestly -24,0,0,0,0,0 and even -16,0,0,0,0,0 looks better.

[mdrejhon]

With the virtual display, I'd be concerned about a few things, most importantly how well the thing would perform, in particular outside of the desktop environment (e.g., in a game).

It depends on your GPU performance. The RTX 3000-4000 series can shader-process a 1440p framebuffer in less than 1ms, so the overhead should be manageable for those games where it's more critical (e.g. cartoon style games with lots of yellows). But I'd just have a toggle to enable/disable the filtering, or let all OpenGL software bypass the driver (configurable option).

I realized after posting the above that the subpixel layout is RWBG not RWGB (even though I wrote it correctly above, I didn't really consider it), so my -24,0,0,0,0,0 doesn't quite do what it should. However, it's still significantly sharper than the default.
I changed the layout to -24,0,24,0,-24,0 to swap the G and the B as well and the results are below.

Yes, it's RWBG.

You might want to make it spatial compensated, where center of R pixel is (25%+25%/2) = 37.50% offset to left, B pixel (25%/2) = 12.5% offset to right, and G pixel (25%+25%/2) = 37.50% offset to right. Basically accomodate that bigger gap between R subpixel and B subpixel. However, there are some side effects of the various decisionmaking on this.

Centre of R subpixel = 1.5 subpixel-width offset to left of pixelgroup center
Centre of W subpixel = 0.5 subpixel-width offset to left of pixelgroup center
Centre of B subpixel = 0.5 subpixel-width offset to right of pixelgroup center
Centre of G subpixel = 1.5 subpixel-width offset to right of pixelgroup center

Which translates to percentages relative to entire pixelgroup width:

Centre of R subpixel = 37.50% offset to left of pixelgroup center
Centre of W subpixel = 12.50% offset to left of pixelgroup center
Centre of B subpixel = 12.50% offset to right of pixelgroup center
Centre of G subpixel = 37.50% offset to right of pixelgroup center

Even treating RWBG as RBG is better than nothing, though I noticed slight improvement when treating it as R_BG being aware that R-B coupling is wider than B-G coupling, and just letting the panel decide on how to illuminate the W subpixel. Some minor side effects from that, but some experimentation needed software-side.

Try a new line that factors in a gap caused by not being able to control the W subpixel directly;

[mdrejhon]

Debug Page For Subpixel Developers & GEOMETRY Tweaking

(Good For Boardroom Meeting Demos Too)

BTW, for validation testing, I have a test page for you subpixel software developers.

https://blurbusters.com/files/test/text-test.html

Thumbnail:

View the original page to utilize your subpixel rendering.

Also, I baked-in the subpixel rendering in this sample image:

[mdrejhon]

Slight Further Improvements Due to W Subpixel Being Fatter

Fix for Pink Tinting to White Text on Black Backgrounds

There are minor variants that improve certain modes better (e.g. Dark Mode) than others (e.g. Light Mode) compensating for the fact that some WOLED subpixels are very slightly bigger than others, but this is generally the most generically accurate:

Centre of R subpixel = 37.50% offset to left of pixelgroup center (-24)
Centre of W subpixel = 12.50% offset to left of pixelgroup center (ignore)
Centre of B subpixel = 12.50% offset to right of pixelgroup center (+8)
Centre of G subpixel = 37.50% offset to right of pixelgroup center (-24)

It is my understanding that PixelLayout defines pixel centers for Rx,Ry,Gx,Gy,Bx,By to represent subpixel centres within a cartesian 64x64 grid from [-32,-32] thru [+32,+32], so you scale accordingly.

This is precisely why the best historical LG WOLED configuration line has been traditionally this:

PixelLayout=-24,0,24,0,8,0

However, the subpixels vary slightly in width on some WOLEDs, and so, optimal pixelcenters may vary slightly, if you want to manually tweak the pixel centers a bit. For the fuller context, I would like to crosspost this tip, for people who want to manually tweak some improvements, to see if it helps.

The fact that the W unfiltered subpixel is larger, pushes the R slightly to the left, and B, G slightly to the right, so you may want to experiment with numbers such as:

PixelLayout=-26,0,26,0,10,0

By understanding this science, try to fudge your "24" about 1,2,3,4 units bigger away the zero origin, while fudging the "8" similar units bigger (or a bit less). See if it eliminates the "pink-color" tinting, when you do this.

You may need to experiment with asymmetries like [-27,0,26,0,10,0] or [-27,0,28,0,11,0] to properly align with the asymmetric-widths of all those subpixels. Very fiddly, but you could ultra-macro-zoom the photograph, crop out one pixel, scale to 640x640 in a paint app such as PaintNET or similar, and mousearrow-center those subpixels, and math out those ideal subpixel-center coorinates. Or design an AI and/or logic algorithm to do this automatically, just supply a macro photo and you get automatic PixelLayout. Voila?

The further improvement is very subtle (not noticed by all), but is there for at least some content, if you compensate for the fact that some WOLED subpixels are slightly asymmetric (<10%) to accomodate the differing special efficiencies of different primaries. This was a tweak to brighten OLED for a given wear-and-tear, by milking the spectral efficiencies better.

This will vary from WOLED to WOLED and some of them have much fatter W subpixel than others, so you may have to vary more to kill the pink-color tinting for white text on black backgrounds. The pink tinting (a combo of unbalanced R+B tinting to white edges) is partially caused by the the R+B subpixels (adjacent to a very fat W subpixel) not having perfectly centered coordinates

[Sporesirius]

Hi, this is really nice. Is there any effort to push the changes upstream?

I was also thinking about this whole situation with the different subpixels. Wouldn't it be possible for the industry to add new metadata to EDID or the successor of EDID the VESA standard DisplayID, something like "subpixel_geometry", so that the OS can detect this and adjust accordingly to the monitor's subpixel geometry?

EDIT: I wrote an email to VESA about this.