Increasing BPP, at least for primary fonts

[owenfromcanada]

I'm relatively new to the PineTime dev community, so I'm not sure if this has been discussed before. But has there been any consideration on improving the font anti-aliasing by increasing the"bpp value in fonts.json?

What I've tried so far: increasing bpp to 4 for the jetbrains_mono_bold_20 drastically improved the look (I removed the patches for the 0 and M characters, no longer needed), which increased flash usage by 13.9kB (about 3.6%). But I haven't checked whether this changes battery consumption or anything.

Any thoughts or history on this?

[mark9064]

Sounds interesting, could you post some comparison shots?

InfiniTime has been more tight on flash space than it is now as a lot of resources have been moved to external flash. So there might be some space for something like this, but it would have to be weighed up against the ability to include new watchfaces, apps etc.

[mark9064]

Great analysis, thanks :) 2bpp on the main system font definitely looks possible, from the shots here I agree it looks much better. Maybe for some of the other smaller fonts 2bpp would make sense too - the icons look nicer IMO (lv_font_sys_48) and I imagine the weather icons do too.

I've just had a look and despite the name, none of the fonts use LVGL compression. I'm not sure why, maybe there is a good reason, but perhaps that could be enabled on bigger infrequently used fonts e.g the PTS font and that may allow 2bpp without it becoming huge? Would need some testing to see if performance is still OK (and whether it works at all!)

[owenfromcanada]

I found some time to experiment a bit with compression.

It turns out that compression doesn't do anything on 1bpp fonts. But with 2bpp and 4bpp, the compression makes a bigger difference, especially with larger fonts:

Setting	Flash Size	Delta	Percentage	Delta
all fonts 1bpp (current setting)	379kB		79.9%
all fonts 2bpp (uncompressed)	402kB	+23.0kB	84.7 %	+4.85%
all fonts 2bpp (all compressed except jetbrains_mono_bold_20)	381kB	+2.2kB	80.3 %	+0.46%
all fonts 2bpp (all compressed)	379kB	-0.2kB	79.8 %	-0.05%
all fonts 4bpp (uncompressed)	455kB	+75.8kB	95.8%	+15.96%
all fonts 4bpp (all compressed except jetbrains_mono_bold_20)	401kB	+21.9kB	84.5 %	+4.60%
all fonts 4bpp (all compressed)	395kB	+15.9kB	83.2 %	+3.34%

For the record, I also tried compression on the PTS font individually. At 4bpp compressed, there was only a difference of +188 bytes compared to 1bpp.

Apparently bumping up all fonts to 2bpp and using compression somehow results in a smaller flash size? If anyone wants to try replicating, it's just a matter of adding "compress": true into each font in src/displayapp/fonts/fonts.json (and changing the bpp values).

The lvgl documentation notes that compressed fonts are about 30% slower to render (which is why I tried omitting the "main" font). I haven't had a chance to test on hardware to see if there is any noticeable impact, though the simulator seems fine with it.

I'm not sure when I'll be able to test on the hardware, so if anyone else wants to give it a spin, that would be great! Otherwise I'll report back when I have access to some hardware.

[mark9064]

Alright so I took a look. LVGL font compression requires decompressing entire glyphs into memory. For large glyphs like open_sans_light, it's impossible even at 2bpp without running out of memory or being nearly exhausted / struggling due to fragmentation. Rendering speed is also low and decreases as the glyphs get larger. I'd say all the large fonts can't be compressed. Decompressing them into memory isn't feasible and performance is too poor.

I think to minimise surprises we should probably avoid compression, as the runtime memory cost might cause problems in certain scenarios.

A good compromise to me seems to be 2bpp on jetbrains_mono_bold_20, jetbrains_mono_42 and fontawesome_weathericons. These are the three smallest fonts and I think they have the most to gain with AA while not blowing up in size too much.

What do you think?

[mark9064]

There's also #1937 to consider which I haven't tried. Would be interesting to see what the flash usage is like with this

[mark9064]

A good compromise to me seems to be 2bpp on jetbrains_mono_bold_20, jetbrains_mono_42 and fontawesome_weathericons. These are the three smallest fonts and I think they have the most to gain with AA while not blowing up in size too much.

I've been running this on my watch the past week and it's been really nice. Still interested to hear your thoughts :)

[kieranc]

I've been running 2bpp for most fonts and 4bpp for weather icons and jetbrains_mono_bold_20, no compression for about a week now. It looks great and doesn't seem to have had any negative effects. I've also been running it with Subpixel rendering for extra smoothness.

[kieranc]

I think they do have to be regenerated but it happens at compile time, this PR only affects the fonts stored on internal storage not ext flash so regenerating them is a non issue. --lcd is an option for lv_font_conv to enable subpixel rendering : https://github.com/lvgl/lv_font_conv#cli-params

[mark9064]

OK gotcha. I'm not seeing subpixel: true set on any of the fonts though, so won't it be the default value which is off (subpixel:bool=False)? Sorry if I'm being dense here :P

[kieranc]

You should always assume it's me being dense first :) I hadn't understood well enough that the subpixel stuff also needs enabling in fonts.json, I'll make a new build and see how it performs.
This does mean the bpp changes alone make a significant visual improvement!

[kieranc]

Oof, so, subpixel hinting has a significant cost in flash usage. I just tried enabling it just for jetbrains_mono_bold_20 and flash usage went from 87.93% to 96.22%. It's less significant at lower bpp levels but i don't think we can enable both.

[mark9064]

Yeah I guess that makes sense as it means that each pixel must be stored 3 times for each colour channel
Maybe we skip subpixel rendering then and just focus on 2/4bpp? Or maybe 1bpp with subpixel looks better than 2bpp?