It's 11 PM. I've just finished what feels like the a good starting set of controls for my .NET Win32 dialog wrapper library. To wrap it up, I want to write some sample dialogs for testing and showcasing purposes. I start off with a WinForms-like "oops you got an exception" dialog, for unhandled exceptions from dialog procedure handlers.
Dialog definitions use dialog units (DLUs), which scale with the dialog's font, which scales with system font. That allows for automatic scaling, but it requires you to work with these DLUs instead of pixels. But no biggie.
I'm writing a dialog that has the size (200, 100), and its controls collectively occupy the rectangle (4, 4, 192, 96). A nice 4-DLU-padded working area.
The only thing is, when I run the dialog, it goes bananas:
Well that's mighty weird. I thought that using DLUs, especially in multiples of 4, would give me beautiful scalable layouts.
I go to my sandbox VC++ project (because Visual Studio doesn't want you to create an .rc file in a C# project, although it will let you if you create a file with that extension manually, but it'll still shower you with errors if you try to open it), drop a Static control on the dialog box, then open the .rc file in the text editor and modify the dialog's and control's bounds. I open the dialog editor again, and the control looks just as expected - centered, with a nice padding.
I check if the font makes a difference - the dialog editor uses MS Shell Dlg 8, and my library uses the system's message box font (as a good boy dialog library should!), but inserting Segoe UI 9 into the dialog editor didn't reproduce the issue.
I notice that the .rc file uses DIALOGEX - that's the extended template. My library uses the basic one because I'm not interested in the extra features/complexity. I'm replacing DIALOGEX with DIALOG in the .rc file, but the dialog editor re-saves the dialog as DIALOGEX anyway. And judging by Microsoft's documentation, coordinates in both formats work the same way, so there shouldn't be any discrepancies.
I turn to ChatGPT (which I couldn't have done all the work so far without), and it gaslights me into thinking that, despite how DLUs are supposed to work, it's all due to integer rounding and control borders that the dialog manager doesn't account for. I would take that answer, but it 1) wildly contradicted my belief that dialog boxes are supposed to make my life easier than using CreateWindowEx, and 2) meant I'd have to bother writing custom geometry logic, which I don't want to.
Out of options, I write a piece of code that stacks 10 10-DLU-high buttons on top of each other inside a 100-DLU-high dialog. And I see what seems like an explanation:
It feels like dialog items use client coordinates, but the dialog's own size translates to window coordinates.
But that shouldn't happen! Dialog editor doesn't have this issue, and ChatGPT suggests that all dialog units should indeed correspond to client area coordinates. And it just wouldn't make sense Win32-design-wise to introduce a separate coordinate system into this deal.
Finally, I take a look at my default window styles for my dialog boxes:
public uint Style = WS_SYSMENU | WS_MINIMIZEBOX | DS_MODALFRAME | DS_SETFONT | DS_CENTER;
public uint ExStyle = WS_EX_APPWINDOW;
I remember how I picked those styles - I figured that WS_OVERLAPPEDWINDOW would be too much for me, since it included a resizeable border (big no for a templated dialog) and a maximize button (same thing). So, I looked at the styles that comprised WS_OVERLAPPEDWINDOW:
public const UINT WS_OVERLAPPEDWINDOW = (WS_OVERLAPPED | WS_CAPTION | WS_SYSMENU | WS_THICKFRAME | WS_MINIMIZEBOX | WS_MAXIMIZEBOX);
Microsoft's Window Style documentation made little sense - style descriptions seem to had been relevant for Windows XP, but not for Windows 11. None of the "border"/"frame" styles affect the window's border appearance, and the window still has a caption bar with and without WS_CAPTION. I know that because I went through the trouble of writing a custom utility that lets me test various window style combinations for myself, no thanks to MSDN.
I erase everything from Styles in hopes that I'll be able to solve the issue and re-add styles one by one until I find the problematic one. But no - the issue remains.
I try to mimic a "normal" window and replace all styles with WS_OVERLAPPEDWINDOW. Ooh, now I see all 10 buttons! But in a Windows 95 font.
Could it be that the mere usage of DS_SETFONT makes the dialog manager go crazy? Well, no - if I restore my initial style set, but remove DS_SETFONT, I see the same wrong layout with the same Windows 95 font.
At this point, I'm looking at the styles that combine into WS_OVERLAPPEDWINDOW and inserting the missing ones one by one.
The issue was gone when I added WS_CAPTION.
I was fascinated and frustrated.
On one hand, RTFM, right? I should've read Microsoft's window style docs and picked the styles accordingly.
On the other hand - those window style docs are unusable, and I didn't just want to trust what I can't understand or choose what seems right - I had to test them for myself. And I picked the set of styles that both seemed right, and worked well enough. It was minimal and it did the job. WS_CAPTION was not one of those styles - the window had its caption either way.
The lesson is - the purpose of WS_CAPTION, in Windows 11, is to make sure that the dialog bounds in your dialog template don't accidentally apply to the window area, rather than the client area. That's all it's good for. I guess the dialog manager actually checks for that style, for some reason, rather than retrieve the window's client coordinates. Or maybe it's a non-dialog specific behavior where the client area isn't offset by the caption size by the time the dialog manager creates its controls.
I don't know, and I don't think I care - I got this silly bug fixed, and now that I've vented, I'm going to sleep.