Enaml - part 2

Enaml - part 2

in

This is the second post about my Domain Specific Language (DSL). See part 1 if you missed it. There’s also part 3 which I separated into a separate post since this got too long.

A quick reminder is that an Enaml document is a tree of nodes, and each node can have text and/or tags.

Basic tags

These are the basic tags that structure, name, and populate the nodes. Later we will look at more advanced meta tags that manipulate other nodes or even the parser.

Key tag

The key tag is used to name the node and uses the colon:

:entities.lakes.ontario

This is obviously inspired by keywords, which are often used to define keys in EDN files.

Unique keys and IDs

Remember that in Enaml the document is a tree and each node has a locally unique id (no sibling nodes may have the same key). This means that if you take the keys at each step from the root to the node, you get a unique id for the node itself.

You can see how useful this is if you’ve ever worked with React and gotten those “key missing” errors because React can’t deduce which elements were changed. The lake engine (I guess I am calling it that now) also works similarly so having this whole category of issues automatically sidestepped is pretty nice.

Paths

The way the key tag works under the hood is that it actually defines a subpath to the node from its parent. So these four examples are the same from the parser’s point of view and create the same three nodes:

:entities
  :lakes
    :ontario "Lake Ontario"
:entities.lakes
  :ontario "Lake Ontario"
:entities
  :lakes.ontario "Lake Ontario"
:entities.lakes.ontario "Lake Ontario"

As you can see, this helps an author reduce the space needed even with linebreaks and indentation mattering in Enaml. It is also useful for modding since the modder can just add children to an existing node without modifying the original file.

This relates to the second useful thing about the key tag, which is that the path it defines can be used for inheritance or overwriting tags. More about these in the meta tag section below.

Data grouping

The third useful thing about the key tag is that it allows an author to group the data however they want. You can either group data by type (GUI, events, entities, etc.) or by domain (main menu, town exploration, combat, etc.). The end result from the game’s point of view is the same.

An example is probably needed. These two examples both result in the same data structure. Here’s the plain structure:

# All entities
:entities
  :lakes
    :ontario "Lake Ontario"
    :algae "Algae Lake"
# All actions
:actions
  :fishing
  :research

And here’s the same data grouped by domain:

# Canada
:entities.lakes.ontario "Lake Ontario"
:actions.fishing
# Antarctica
:entities.lakes.algae "Algae Lake"
:actions.research

(By the way, look at this photograph melt pond image on Wikipedia I came across while looking for a list of lakes. Damn.)

Obviously the data would need conditions and other things for the engine to understand the domain divisions but this is something that would have to be done anyway so I’ve simplified the example to hopefully make it clearer.

This also in the same file. In a real use case you would probably have one file for Antarctica and another for Canada. And separate files for common stuff shared by both.

Attribute tag

The attribute tag defines the values associated with the node, like the values for a GUI element:

.align.left .border.width.2 .on-click.invite.PLAYER

Above we have a node with four attributes. Basically you can think of these as XML attributes or EDN hash-map key-value pairs.

Some of the parts of the attribute tag define the path to an attribute and some (usually the last part) determines the value.

So here the above values would result in the following Clojure data structure:

{... ; other node related stuff
 :align    :left
 :border   {:width 2}
 :on-click [:invite '?PLAYER]}

On schema

But how does the system know when the path ends and the value begins?

This is one of the joys of a DSL. I was wracking my brain trying to figure out how the syntax is supposed handle this problem. I was almost resigned to adding some sort of comma based system to the notation which would have been pretty horrible to read and write.

But then I realized that precisely because I am not making a generic language, I already know what schema the system is using. So the user never even has to know what the above looks like in the code. They just have to know where and how to use it.

This doesn’t just apply to attributes; every tag is simply a prefix character followed by dot notation alphanumeric, kebab-case words. Sometimes with syntactic sugar on top.

Condition tag

The condition tag starts with a question mark or an exclamation point and defines a pre-condition for the node:

?ITEM.type.potion

Would make the node valid when the variable ITEM was an entity that was a potion. The negation of that condition would use an exclamation point and be valid if the ITEM was not a potion or if it didn’t have a type at all:

!ITEM.type.potion

The meaning of “valid” here depends on the context, so a UI widget would only be shown on the screen if it was valid. An event listener or guard would only trigger if it was valid too.

One way of looking at it is that a node with an invalid condition doesn’t exist in the tree at that point. This includes its child nodes.

Datalog

Like I said last time, under the hood the conditions become Datalog (specifically DataScript) query clauses. It’s a pretty natural fit with the dot notation since Datalog clauses are also mostly a sequence of terms.

For example, the above condition (the question mark one) becomes:

[?ITEM :type :potion]

However, there are special forms that allow the user to go beyond simple entity-attribute-value comparisons. These are necessary because more advanced forms of clauses in Datalog have some funky nesting and symbol use involved which would be hard to do justice to with dot notation.

This is true of the negation first of all. The above negation would become:

(not [?ITEM :type :potion])

Special conditions: case

The most important of the special conditions is the case condition:

?IMAGE.case.MENU.main.bg1.options.bg2.bg3

Which is read as “If we are in the main menu, use background image 1, if we are in the options menu use background image 2, otherwise use background image 3.”

It’s probably easier to understand through its rough Clojure equivalent, which would be something like:

(let [IMAGE (case MENU
              :main    :bg1
              :options :bg2
              :bg3)]
     ...)

Although of course what it is actually turned into is Datalog where clauses. Behold:

[(or 
   [(and [(= ?MENU :main)] 
         [(ground :bg1) ?IMAGE])]
   [(and [(= ?MENU :options)] 
         [(ground :bg2) ?IMAGE])]
   [(ground :b3) ?IMAGE])]

Now there are other ways of doing the above (and I haven’t actually checked that it works), but I hope you see how having a DSL for these is useful for shielding the user from the Datalog syntax.

To be fair to Datalog, this use case is far from normal since we have to bind the “output” of the condition to a variable to be able to use it in the rest of the node. If you were making the queries directly this would not be necessary.

Also, the fact that Datalog has such a composable structure is the reason why this is so easy to build on top of it in the first place.

Special conditions: if

Another important special condition is if, which is really just a special case of case. You can probably figure out how this checkbox logic works based on the above:

?IMAGE.if.CHECKED.box-ok.box-empty

In addition to the simpler syntax, this also solves another problem: we can use the else part of the condition to handle missing values. Normally a Datalog query will fail if we are trying to bind a value that does not exist. Using the if and case allows us to sidestep that issue entirely.

Tag joining

Some of you may have noticed that these are different from how special predicates are usually done in Datalog. Specifically, the predicate (if or case) is the second term of the clause and not the first. Idiomatically it should be the other way around.

There are a couple of reasons for this. First, Datalog clauses typically don’t have separate output bindings so those are a bit clearer if they are outside the body of the special condition.

Secondly and more importantly, this puts the output variable at the beginning of the tag. This allows us to add some syntactic sugar to make the user’s load a little lighter with tag joining.

Consider this extended checkbox example:

.image.IMAGE ?IMAGE.if.CHECKED.box-ok.box-empty

Notice how the user has to separate the image selection logic from where it’s used? How they have to name the variable even though they use it once?

This is why in Enaml there is a special syntax for joining tags. It uses the % character to omit the variable so this is the equivalent of the above:

.image.%.if.CHECKED.box-ok.box-empty

Which removes 11 characters, has no need to come up with a variable name and visually connects the two pieces of data.

Effect tag

On to the effect tag, which determines post-conditions for the node:

*toggle.SYSTEM.sound-on

They are similar to (pre-)conditions that they are ultimately turned into Datalog, but in this case datoms. Where pre-conditions read the database, effects write to it.

The first term is always the type of effect and the rest are parameters. Here the toggle effect sets an entity attribute to true or false, which is useful for checkboxes and flags.

Addition and retraction

There are two syntactic sugar variations on the effect, addition:

+PLAYER.mood.angry

Which turns into:

[:db/add ?PLAYER :mood :angry]

and retraction:

-PLAYER.mood.angry

Which turns into:

[:db/retract ?PLAYER :mood :angry]

You can also retract an entire entity if you leave out the rest of the tag, like so:

-PLAYER

Together with conditions there’s obviously a lot here Enaml can do to reduce boilerplate by making special conditions and effects.

I’m also thinking the user should be able to define their own composite functions as well. But that has not been implemented, at least not yet. They wouldn’t require any special syntax so not relevant here either way.

Conclusion

This post was getting long so I separated the advanced tags into their own post. See the next part for the juicy advanced stuff.