Minute surrounded by colons is treated as symbol

[phormio]

In a time such as 10:13:02, Pandoc considers :13: to be a symbol. See terminal transcript:

bash$ podman run --rm pandoc/core --version
pandoc 3.8
Features: +server +lua
Scripting engine: Lua 5.4
User data directory: /usr/local/share/pandoc
Copyright (C) 2006-2025 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.
bash$ echo The door opened at 10:13:02. | podman run --rm -i pandoc/core --from=djot
<p>The door opened at 10<span class="symbol">13</span>02.</p>
bash$ 

Note that this causes the colons in the time to be lost.

I don't think this treatment of "10:13:02" is the best reading of the "Symbols" section in the Djot syntax reference. Am I wrong?

[phormio]

I have consulted the relevant file in the test suite of djoths.

[jgm]

Agreed, this is very awkward. It's not specific to pandoc or the Haskell djoths library. Same result in the reference parser.

The syntax description says "Surrounding a word with : signs creates a “symbol,”", but it isn't clear about what counts as a word I suppose.

[jgm]

I'm open to suggestions. We could tighten up the requirements for a symbol, e.g. insisting that the colons are adjacent to whitespace, end of line or punctuation. Any change like this makes the syntax more complex and reduces the flexibility of symbols. Alternatively, we could just ask people to put a backslash in there if they don't want the symbol. (But this is a context where one wouldn't think one needs to do anything special.)

[vassudanagunta]

Is disallowing purely numeric symbols a practical solution or a hack asking to be exposed by a non-numeric counterexample?

[phormio]

Is disallowing purely numeric symbols a practical solution or a hack asking to be exposed by a non-numeric counterexample?

I think IPv6 addresses would count as a non-numeric counter-example. (Unless 'numeric' is defined as "consisting only of hexadecimal digits".)

[Omikhleia]

I think IPv6 addresses would count as a non-numeric counter-example. (Unless 'numeric' is defined as "consisting only of hexadecimal digits".)

URNs too, but that might be regarded as an edge case (i.e. assuming one doesn't want them as verbatim code...)

[jgm]

Is disallowing purely numeric symbols a practical solution or a hack asking to be exposed by a non-numeric counterexample?

One of the motivations for symbols was to provide a syntax for emojis -- and we have things like :1: and :100: with the usual emoji aliases.

[jgm]

Proposal:

• limit symbol characters to ASCII alphanumerics [EDIT:] plus certain symbols, e.g. +, which occurs in some emoji names, _, -, maybe & and # (see [Proposal] more permissive definition of allowed symbol syntax #296, where the following is proposed: [A-Za-z0-9._&+*#-])
• a symbol may not be followed directly by a character from this class

This handles times (11:00:00) and ipv6 (2001:0db8:85a3:0000:0000:8a2e:0370:7334). Anything else we'd be missing?

(Motivation for limitation to ASCII: testing for alphanumerics more generally requires unicode-awareness, which we're trying to avoid requiring to make parsing easier. And I think this is all we need for symbols.)

[jgm]

More concisely: A symbol is :[A-Za-z0-9._&+*#-]+:, not followed by [A-Za-z0-9._&+*#-].

EDIT: I guess this may be too restrictive for some desirable uses of symbols, e.g. one might want an emoji followed by a letter, conceivably? We could perhaps permit a braced form for those cases, {:symbol:}.

[phormio]

That rule fails on:

We've arranged a meeting at 12:00: our last meeting in Nashville.

[jgm]

One possible solution (though backwards incompatible) would be to require symbols to be braced: {:100:}. A less drastic change would be to allow the unbraced format in certain cases.

[vassudanagunta]

another option would be a symbol syntax that is highly unlikely to ever collide with non-symbol intent. e.g. ::100::.

[jgm]

Thinking out loud:

symbolchar = [A-Za-z0-9_&+*#-]
symbol = ':' symbolchar+ ':' not immediately followed by [A-Za-z0-9]
                |  '{:' symbolchar+ ':}'

This allows:

My:+1:.
:+1:-raters.

but requires:

{:+1:}raters

Times and ipv6 addresses are handled as expected.

Unlike the :: proposal, this makes it easy to write symbols in normal cases.