Question: Design of Uniform Representation/Model

I brought this up at IRC but didn't get any comments (maybe I posted at the wrong time).
I paste it here:

I just found out that prompter does accept HTML string, and I
can format my docstring by writing HTML in it I'm not sure if
that is intentional design or accidental, though. Interpreting
strings as raw HTML strings without escaping might cause
problem (e.g. what if a "<" just shows up in the docstring of
CL:<)?
There are several intentional design I can see:

  1. Use plump node uniformly. String has to be a special case and is
    (a) escaped or (b) never escaped.
  2. Use spinneret style sexp uniformly. String is naturally a subset
    of spinneret style sexp and is escaped.
  3. Status quo, use unescaped HTML string uniformly.

I like 2 the most, there's a specific problem though: for
docstrings, an S-expr won't be recognized as documentation in place
of the docstring maybe we can add a convention: if we see a quoted
form as the first form of a function/command/etc body, store it as
structured documentation?

I think this is a very important question. A dominating ontology is what usually defines the characteristic of an operating system/environment.

  • UNIX: hierarchical file system and files (unstructured bag of bytes)
  • Windoze: wIndOWs and a UNIX-like FS
  • Emacs: Buffer and text /with text properties/ (a bit more structured, but still not very structured).

What would be a dominating ontology for Nyxt?

If we choose option 2 (S-expr), I think we could also get rid of plump dependency completely. We’re not using it to parse HTML anyway (we’re getting it from JSON), and we can replace the DOM operations with simply list operations on S-expr.

That's a hard question. I've been postponing answering it, and I will not even try to give a definitive answer here, as I cannot tell for the whole team.

I personally like Plump and extremely structured/typed data in general.

We're not using it to parse HTML anyway

As a response to this, I must say that Plump is used as the basis for all the element-hints. All the code is built on top of method dispatching on tag classes. Even though we don't plump:parse anything directly, Plump is still vital for all the document model handling we have. Re-implementing Plump functions on top of s-exps would be a lot of work, and is not likely to bring much benefit.

Rewriting the code to use Spinneret-style sexps will make things too low-level and bloated.

The unescaped HTML point is totally valid, we should do something about it! The introduction if this raw HTML option was intentional, to allow prompt-buffer styling unrestricted by anything. But there indeed could be problems with it.

Structured/typed data is good. I'm not a big fan of Plump's api which basically just mirrors JavaScript DOM API, it's a pain to work with and makes heavy DOM manipulation code ugly. You can't cons Plump nodes.

I'd argue s-expr is almost equally strongly structured/typed. The "syntactic" type is simply determined by its operator. And spinneret S-expr has some additional niceties, like each :class is separate. In Plump one has to concatenate strings, eww.

The higher level question is: do we want an S-expr operating environment, or a CLOS operating environment? I've heard the wisdom from older Lispers: CLOS implies a remarkable ontological commitment, which is useful for particular cases but not a good idea as a universal model. Even if we take the CLOS route, the current plump interface is not CLOSy -- e.g. the attribute hashtable should probably be reworked into extensible objects or prototypes.

Plump is still vital for all the document model handling we have. Re-implementing Plump functions on top of s-exps would be a lot of work, and is not likely to bring much benefit.

An immediate benefit is that DOM manipulating (rather than mostly just reading) code can be much cleaner, and make use of standard Lisp vocabulary. In the long run, I think a uniform model will benefit a lot for an operating environment. S-expr is already the "native" format for code in Lisp, why have a different format for structured documents? At the moment a unfirom model means we can easily embed document in our code (e.g. as structured docstring). It's much more important after we have a structured code editor, under a uniform model it can immediately turns into a document editor.

Rewriting the code to use Spinneret-style sexps will make things too low-level and bloated.

Things don't need to get any low-level at all, we have one thing called abstraction :).
And we just need a (head :input) specializer, which is built-in in Emacs and, of course, can be added to CLOS via MOP. The whole current DOM access interface can be reprovided via S-expr.

I can take the task to reimplement those, if desired.

But maybe this does not need to be done immediately. What is working could continue to work, and the structural editor would need to maintain its own S-expr DOM (because after some trials I'm convinced plump is not a sane interface to build a Lisp editor on). We would momentarily keep two representation of the same data in the Lisp image, which is not nice, but not an immediate disaster either.