objective c - Faking Subpixel Antialiasing on Text with Core Animation

Question

For anyone working on a project with Core Animation layer-backed views, it's unfortunately obvious that subpixel antialiasing (text smoothing) is disabled for text not rendered on a pre-set opaque background. Now, for people who are able to set opaque backgrounds for their text (either with a setBackgroundColor: call or the equivalent setting in Interface Builder), this issue doesn't present too much of a problem. For others, though, who have absolutely no way to work around it, it's not something one can ignore.

I've been looking online for well over two days for a solution, and nothing usable has come up. All I want to do is create a text label (not user-editable) that is set on a sheet window (sheet window backgrounds are transparent, so having the sheet's content view declare wantsLayer as true disables text smoothing for all labels on the window); something very simple. I've seen a lot of contested solutions (a quick Google search will bring up this topic in many other places), but so far, all of those solutions rely on people being able and willing to compromise with an opaque background (which you cannot use on a sheet).

So far, the best direction I can imagine taking this in is pre-rendering the text with text smoothing onto an image, and then displaying the image as usual on a layer-backed view. However, this doesn't seem to be possible. The 'normal' way I would assume one would try is this:

NSAttributedString *string = [[self cell] attributedStringValue];
NSImage *textImage = [[NSImage alloc] initWithSize:[string size]];
[textImage lockFocus];
[string drawAtPoint:NSZeroPoint];
[textImage unlockFocus];

But that doesn't seem to work (most likely because when called from drawRect:, the graphics context set up already has text smoothing disabled - subpixel antialiasing is turned off, and regular antialiasing is used instead), but neither does the more involved solution found in this question (where you create your own graphics context).

So how is doing something like this possible? Can you somehow 'fake' the effect of text smoothing? Are there even hack-ish workarounds that will get something set up? I really don't want to have to abandon Core Animation just because of this silly issue; there are a lot of benefits to it that save a lot of time and code.

---

Edit: After a lot of searching, I found something. Although the thread itself only reaffirms my suspicions, I think I may have found one solution to the problem: custom CALayer drawing. Timothy Wood, in one of his responses, attached a sample project that shows font smoothing using several techniques, several of which work. I'll look into integrating this into my project.

Edit #2: Turns out, the link above is a bust as well. Although the methods linked give subpixel antialiasing, they fail as well on transparent backgrounds.

Answer 1

If you're using a transparent sheet, you don't know in advance what the pixels below it will be. They may change. Remember that you have a single alpha channel for all three colors: if you make it transparent, you won't see any subpixel effect, but if you make it opaque, all three subelements are going to get composited with the background. If you give an edge the right color for compositing over a white background, it won't look right if the background changes to some other color.

For example, let's say you're drawing black text on a white background, and the subelement order is RGB. A right edge may be a faint blue: high B value (full brightness on the side away from the glyph), slightly lower but still high R and G values (lower brightness on the side closer to the glyph). If you now composite that pixel over a dark gray background, you're going to make it lighter than it would have been if you had rendered black text on dark gray background directly.

Basically, you are not facing an arbitrary limitation of CoreAnimation: it simply makes no sense to use subpixel rendering on a transparent layer that might be composited over an arbitrary background. You'd need a separate alpha per color channel, but since the pixel format of your buffer is RGBA (or ARGB or whatever it is), you can't have it.

But this leads us to the solution. If you know that the background will remain the same (eg, the sheet displays over a window whose contents you control), then you can simply make your layer opaque, fill it with a copy of the covered region of the background window, and render subpixel-antialiased text on it. Basically, you'd be precompositing your layer with the background. If the background stays the same, this will look identical to what normal alpha compositing would do, except that you can now do subpixel text rendering; if the background changes, then you'd have to give up on doing subpixel text rendering anyway (although I guess you could keep copying the background and redrawing your opaque overlay whenever it changes).