I saw this post on my feed, and it caught my eye because I self-host a Mattermost instance. The fact that it has possible critical security issues is a concern for me. So I checked the github issue with the details.

Of course, it’s fully LLM generated, but that’s to be expected at this point. So I opened the first issue described there, a CRITICAL VULNERABILITY no less. It’s the possibility for XSS in email templates. Ok, sure, let’s dig deeper. This post will have a technical part, where I dig into the code; and a more sociological part, where I dig into other things.

Technical Part#

The problematic code is this:

func prepareTextForEmail(text, siteURL string) template.HTML {
    escapedText := html.EscapeString(text)
    markdownText, err := utils.MarkdownToHTML(escapedText, siteURL)
    if err != nil {
        mlog.Warn("Encountered error while converting markdown to HTML", mlog.Err(err))
        return template.HTML(text)
    }
    return template.HTML(markdownText)
}

The return inside the error checking is for the unescaped text, and ideally should use escapedText. But is it possible to even reach this situation? For that, the function utils.MarkdownToHTML() would need to error out.

And can it? Let us have a look:

func MarkdownToHTML(markdown, siteURL string) (string, error) {
	// Turn relative links into absolute links
	absLinkMarkdown := relLinkReg.ReplaceAllStringFunc(markdown, func(s string) string {
		return relLinkReg.ReplaceAllString(s, "[$1]("+siteURL+"$2)")
	})

	// Unescape any blockquote text to be parsed by the markdown parser.
	markdownClean := blockquoteReg.ReplaceAllStringFunc(absLinkMarkdown, func(s string) string {
		return html.UnescapeString(s)
	})

	md := goldmark.New(
		goldmark.WithExtensions(extension.GFM),
	)

	var b strings.Builder

	err := md.Convert([]byte(markdownClean), &b)
	if err != nil {
		return "", err
	}
	return b.String(), nil
}

As you can see from this, the only place where this function can return an error is when it calls md.Convert(). This is part of the of the goldmark package, so we need to look there. The Convert() function is defined in the markdown.go file:

func (m *markdown) Convert(source []byte, writer io.Writer, opts ...parser.ParseOption) error {
	reader := text.NewReader(source)
	doc := m.parser.Parse(reader, opts...)
	return m.renderer.Render(writer, source, doc)
}

So, the only way for Convert() to return an error is from the Render() function:

func (r *renderer) Render(w io.Writer, source []byte, n ast.Node) error {
    (...)
	writer, ok := w.(util.BufWriter)
	if !ok {
		writer = bufio.NewWriter(w)
	}
	err := ast.Walk(n, func(n ast.Node, entering bool) (ast.WalkStatus, error) {
		s := ast.WalkStatus(ast.WalkContinue)
		var err error
		f := r.nodeRendererFuncs[n.Kind()]
		if f != nil {
			s, err = f(writer, source, n, entering)
		}
		return s, err
	})
	if err != nil {
		return err
	}
	return writer.Flush()
}

Now, we arrived at the meat of the problem. First of all: we need to remember that what was passed into the goldmark package was something of type strings.Builder. Since this doesn’t implement the necessary functions for util.BufWriter, it gets wrapped in a bufio.NewWriter. So, the only way for an error to get returned here is through nodeRenderedFuncs. I went and checked, and it turns out that err is always nil in this particular situation. So: Render() never returns an error, so Convert() never returns an error, so MarkdownToHTML() never returns an error, so the err in prepareTextToHTML() is always nil.

Turns out the error checking is actually dead code!

Now, is this a potential bug? Yes, if e.g. the goldmark package ever changes the way it processes its input and creates the possibility of the conversion erroring out, then it’s possible for an attacker to craft a payload that could trigger this code path and exploit this XSS vulnerability.

And should the devs just have used return template.HTML(escapedText) in that code path? Also yes. Why they didn’t, I’m not really sure. But it’s besides the point, which is: there’s no vulnerability here, let a critical one.

The Sociology of the Hype Beast#

Let me be clear, all of this is just AI slop. The poster, a Google employee no less, is trying to prove he can build a Slack clone using nothing but AI (or something like this, I got to admit that I didn’t really read into it too much). So, he decides to prompt the Claude to check on security issues for his project, and other similar open-source projects. He then goes and uses a false report, that he didn’t bother to check, to create a perception - in this case, how the AI-generated code is actually more secure than code that has been written by humans.

Sure, I didn’t bother to check all the other reports. But if the first one I checked turns out to be wrong, and not just wrong but also incredible sloppy, it doesn’t really fill me with confidence about the rest.

This is the exact same thing that happened with cURL, and has led them to close their bug bounty program. It became infested with AI slop reports that follow this exact pattern: the AI spots some potential problem, claims CRITICAL VULNERABILITY!!!11, and the idiot at the wheel just goes along. Doesn’t check, doesn’t follow through, just follows the rush. So, actually, they’re not even at the wheel, they’re just along for the ride.

And I don’t think this is even the worst problem of this attitude. You go and look at these hype-beasts on LinkedIn, or on X, or on YouTube, and even on the news, and they’re what worries is not the self-promotion, the grifting, the lying. The worst is the second-order effect of having one hype-beast after the other, all claiming some for of good old “AI is 6 months away from X”, with X being some form of a complete revolution of some sector of the economy. This creates such a distorted understand of what is actually happening.

I’m tired. Really tired of this. They’re turning a potential useful tool into a weapon of mass bullshit. This is dangerous, and we’re all just smiling and waving, celebrating these morons who are “red pilling”. This isn’t going to end well.