2024-09-15

[^Footnotes]#

Two things about me. I like Terry Pratchett, and I only started frontend when LLMs arrived to help. These two unrelated traits collided in September 2024.

You see, one of the things I like about Terry Pratchett is the way he uses footnotes. They're scattered across the Discworld novels, injecting comedy and worldbuilding with a mechanism I find to be an entertaining meta-conversation. I picked this up in my own writing, which became a problem when I decided to rebuild my blog from scratch.

Apparently Nextjs MDX processing struggles with footnotes, and I lack the background to understand how to fix it.

Note

See the problem#

Are you a good React engineer? Are you a good Markdown writer? Are you a good person?? Come see the problem - and whether you know how to fix it!

Problem example website: react-mdx-issues.vercel.app/

Problem example GitHub: github.com/HebeHH/reactmdxissues

8 Sept 2024#

I start coding a new blog. Specifically, I ask Claude to code me a new blog. It only takes one prompt to get a genuinely decent setup going, and a bit of back-and-forth gives me a framework I like for the homepage.

Everything is in Typescript, Next.js, and tailwindcss.

9 Sept 2024#

With the homepage done, it's time to start adding and building the articles.

My articles are in Markdown.

Note

Mark down the web#

Markdown is a markup language,[0] like HTML. Unlike HTML, it is in zero ways a web-based language. It's actually becoming pretty common to find on the web - a lot of text inputs will allow for Markdown, like GitHub, but it wasn't made to display in the browser.

To solve this, React uses MDX. MDX lets you use .jsx components in your Markdown, and also does the heavy lifting of transforming your stuff from Markdown markup to HTML markup via .jsx components. Libraries can be added on top of it to give you additional or different .jsx components. You can also include your own MDX Components dictionary that lets you overwrite the default components, or add new ones.

That is currently pretty much the extent of my understanding of MDX[0.6].

I start with my article on RAG since it's got a lot of different elements.

Create an MDXComponents.tsx file and handle headers, centering, paragraph text, code, blockquotes, images, and videos. Blockquotes give me the most trouble, because it's applying the <p> tag styling inside the <blockquote> styling, which messes up my sizing.

10 Sept 2024#

I get to the end of the article and realize the footnotes are not footnoting. They're displaying as regular paragraph text, like so:

Screenshot 2024-09-17 at 10.02.49 PM

Note

Standardization is a lie#

Markdown is quite popular. This is great in some ways[0.5] but not so great in others: it's popular enough to have splintered into subgroups. There is no longer one single standard.

XKCD standards

MDX uses the CommonMark standard. CommonMark does not include footnotes[0.7].

I write using Typora. Typora Markdown does include footnotes.

This is essentially the root of all my subsequent problems: I wrote some Markdown under the impression my Markdown was the same as React's Markdown, and was wrong. Somehow "but it works on my machine" pops up absolutely everywhere.

Knowing very little about frontend dev, I immediately turn to Claude with all my code and a request to update MDXComponents.tsx to handle footnotes. The first suggestion I discard as soon as I see the useState in there, because the little I know about frontend includes the fact that Next.js handles Server and Client components differently, and that I don't know enough about frontend to start messing about with useState when I don't think I need to.

The second conversation, it tells me to use remarkFootnotes in the next.config.mjs alongside a server-side mechanism for handling footnotes in MDXComponents.tsx. It still doesn't work.

Going back to Claude, it tells me:

  1. Verify that remark-footnotes is installed and update the handling
  2. Replace your footnotes in the MDX article with <Footnote id="1" />

I'd prefer not to use a React Component to handle footnotes[1] if I can just use the default markdown format instead, so I decide that doesn't work.

At this point, I realize that remark-footnotes is "no longer recommended for use. It’s still covered by semantic-versioning guarantees and not yet deprecated, but use of this package should be avoided." It recommends using remark-gfm instead.

I add remark-grm to my next config: remarkPlugins: [remarkGfm, remarkRehype].

The results are...

Weird.

Note

Footnotes are broken[1.5]#

At this point, I'm using a simple test markdown:

# Footnotes test Short footnote works.[^1] Longer footnotes will not work, whether inline or end of string.[^2] [^1]: SingleStringFootnote [^2]: No footnote with multiple words will work.

As the text here implies, the footnote somewhat works - as long as the footnote is a single string.

Anything with a longer footnote is processed the same as earlier, with both the reference and the entry being processed as normal paragraph text:

Screenshot 2024-09-17 at 10.32.47 PM

Saying that the single-string footnote 'works' is also burying the lede a bit. The footnote reference (the bit in the text) 'works' in that it is a link. Unfortunately, the footnote entry (bit at the end of the text) has disappeared. Or not fully disappeared: it's gone into the link in the footnote reference. Clicking on the ^1 in the above tries to take me to the link localhost:3000/articles/SingleStringFootnote.

I don't need any frontend knowledge to know that's weird; that's trying to take me to a new page. A page that doesn't exist.

Screenshot 2024-09-17 at 10.39.23 PM

Instead, this footnote should be refering me to another location in the same page - the link should look something like localhost:3000/articles/footnotes#1, with the #{id} indicating which HTML element on the page to go to, by id.

Even better, it would be something like localhost:3000/articles/footnotes#footnote1, so that I could use the footnote id 1 as a header or element id elsewhere in the page without risk of it clashing. I'd even accept a constant link to localhost:3000/articles/footnotes#footnotes to be used for all footnotes, which would then be all stored in the same div.

There are many things that could have worked, but none of them are happening.

I go back to read the remark-gfm page more thoroughly. Unfortunately, my lack of understanding of frontend is made very apparent as I understand nothing. I am now even more confused. I do read through the bugs listed for the micromark-extension-gfm-footnote extension which remark-gfm uses, and none of them match my usecase.

13 Sept 2024#

At my boyfriend's birthday party I accost everyone who's touched frontend development.[2]

I ask each of them how to do footnotes in MDX and React/NextJs. Notable reactions include:

  • You don't.
  • Why are you using MDX in the first place???
  • pitying laughter
  • Just use a React Component.

None of these are helpful. Except the last, but that feels like giving up.

17 Sept 2024#

I get home and start working on transforming the rest of my blog as a distraction.

20 Sept 2024#

Latex is also broken?????

Significantly more broken at that, it's crashing the entire site:

Screenshot 2024-09-17 at 10.39.23 PM

I add in remarkMath and rehypeKatex as instructed. Still no dice. Same error. It seems that the Latex[3] is being read as code, and that code is invalid.

So now it's my birthday, and my present is no footnotes and no latex.[4]

21 Sept 2024#

Back to basics. Let's create a new test repo , trim it to basics, and follow exactly the protocol outlined on the NextJs docs and their example App Directory MDX github.

There are two major differences between my website and the documentation example, possiby because the author of my website[5] is working off an old version of the docs.

  1. Routing: There's 3 ways to use MDX in NextJs: file-based routing, imports, and remote. The first two are server-side, which means the full page can be generally be created at build time and then served directly. Remote MDX is client-side: instead of sorting out how the Markdown should look ahead of time, it will be fetched at runtime and sorted out then.

    1. The docs: largely use file-based routing.
    2. My blog: uses remote.

  2. Personal MDX Components handling:

    1. The docs: provide only one setup for MDX Component handling.

      1. File: a top-level mdx-components.tsx file.
      2. Export: export function useMDXComponents(components: MDXComponents): MDXComponents { return { ...components, }}

    2. My blog: uses what Claude gave me, which is for some reason different.

      1. File: a MDXComponents.tsx file inside the Components directory.
      2. Export: const MDXComponents: Record<string, React.ComponentType<any>> = {}.

I start playing with different configurations in the new test repo.

22 Sept 2024#

Is it a next-mdx-remote issue??

Latex is working in every routing mechanism but remote, which suggests it's a problem somewhere in the parsing there.

Footnotes are still looking the same everywhere.

10 Oct 2024#

House is moved! Only took three weeks.

Took some time to tidy the new repo into something I can send around to ask for help. Quite a lot of time.

I then host it on Vercel.

    0: I was today years old when I got the joke about Markdown being a lightweight Markup language. ↩
    0.5: Like the aforementioned ability to use it in many various places. ↩
    0.6: My understanding might improve by the end of this whole saga. Then again, I might just shove copypasted stackoverflows together until it works, and learn absolutely nothing. ↩
    0.7: As of the time of writing, etc, additional disclaimers, and such ↩
    1: I'm also confused about why it told me to keep remark-footnotes if we're just going to use Remark Components anyway. ↩
    1.5: Yet I have started this blog-diary on how my blog footnotes are broken with footnotes included, showing an overwhelming self-confidence that isn't necessarily deserved. ↩
    2: There's a surprising amount. ↩
    3: Prepare for me to be confused about spelling and call it all of Latex, latex, and LaTeX within the same post. That's the problem with writing over days. ↩
    4: It was actually several very nice handsewn tops, which I suspect will end up getting their own blogpost at some point. ↩
    5: Claude, it's all Claude. ↩