Turn Markdown into a PDF

The first version of the inline parser here turned "2 * 3 * 4 is 24" into an arithmetic expression with an emphasised 3 in the middle, and ate both asterisks doing it. That is the failure every naive Markdown implementation ships with, because the obvious algorithm is to find a marker and then scan forward for the next one. CommonMark solves it with delimiter runs and flanking rules, and the two that matter are short: an opening delimiter is never followed by whitespace, and a closing delimiter is never preceded by it. A lone asterisk with spaces on both sides can therefore open nothing and close nothing, which is exactly the behaviour a multiplication sign or a footnote marker needs. The rule also has to be applied to the run rather than to the character. When the parser advanced by one character after a failed "**", the trailing half of that run was still available to close as a single "*", and "**a ** b" came out as an emphasised "a *". Consuming the whole run on failure is what fixes it, and there is a test named for each of those two cases because both were real bugs rather than hypothetical ones.

The second thing that has to be got wrong first is the position of a list bullet. The obvious implementation records the cursor before rendering an item and puts the marker one line-height below it, which works until an item begins with a paragraph gap, or breaks across a page, or opens with a code block. All three shift the first line somewhere the cursor never predicted, and the bullet ends up floating above its own text. What works instead is to let the drawing code report where it actually put the first line, which is the only place that knows. That reporting has to be a stack rather than a single slot, because a list nested inside a list item opens its own capture while the outer one is still waiting: with one slot, the inner item overwrote the outer answer, so the outer bullet landed on the nested line and its own line got none. Rendering the sample document is what surfaced that, not the test suite, which is the argument for looking at the output rather than at a green tick.

Word spacing exposed a subtler problem, and the fix for it is the reason the files come out about forty per cent smaller. This renderer uses only the fourteen fonts every PDF reader already has, which means the document carries no width table: the reader lays out text using its own copy of Helvetica, and its glyph advances differ from the ones pdf-lib uses to measure by a fraction of a point per character. Placing every word at its own absolute x made that difference visible at every single space, because each gap was computed by one set of metrics and rendered against another. Drawing each run of same-styled words as one string instead lets the reader space the inside of the run, and leaves only the boundaries between styles positioned by us. Measured on a sample page, the gaps went from varying between 1.5 and 3.2 points to being exactly 3.058 everywhere.

The built-in fonts have a harder limit that cannot be engineered around, only reported. They are WinAnsi encoded, which covers printable ASCII, the Latin-1 range and about thirty typographic characters that live between 0x80 and 0x9F: curly quotes, the en and em dashes, the bullet, the ellipsis. Everything else, which includes all of Chinese, Japanese, Korean, Cyrillic, Greek, Hebrew and Arabic, would make pdf-lib throw when it tried to encode the string. Throwing at the user would be one honest answer and substituting silently would be a dishonest one, so the encoder does neither: characters with a real ASCII equivalent are folded (a non-breaking hyphen becomes a hyphen, a fraction slash becomes a slash), anything else is replaced with a question mark, and every distinct replaced character is collected and shown. The alternative is embedding a Unicode font, which costs several hundred kilobytes and a second library, and that is a different trade than the one this tool makes.

Pagination is one object rather than a rule scattered through the block renderers. Every block asks the writer to reserve the vertical space it needs before it draws, and the writer decides whether that means a new page, which is what keeps a heading from being set at the foot of a page with its paragraph overleaf: a heading reserves its own height plus the first line of whatever follows it. Two blocks need more than that. A fenced code block draws its tinted plate per page rather than as one tall rectangle, because a listing that spans a break would otherwise have its background hanging off the bottom of the first page and missing from the second. A blockquote does the same with its left bar, closing it on the old page and starting a fresh one at the top of the new. Table rows reserve per row, so a long table breaks between rows and never through the middle of one.

Table columns are sized from their contents, and when the natural widths overflow the measure the surplus is taken from the widest column repeatedly rather than shared equally. Sharing it equally is the intuitive approach and it is wrong: it squeezes a column containing "Yes" and "No" down to nothing in order to spare a column of prose that had room to give. Each column also has a floor of about four characters, and the loop stops when every column has reached it, which means a genuinely impossible table (twelve columns of paragraphs) degrades to something cramped rather than to something with negative widths. Links are written as real link annotations in the page object rather than as blue text, because a document whose links only look like links is a document whose links do not work, and there is a test that reads the URI back out of the annotation dictionary rather than searching the file for the string: pdf-lib writes compressed object streams, so the text is not in the clear and grepping for it passes for the wrong reason.