Some notes on pattern matching

When I first designed Language 84, I included a traditional pattern matching construct that looks like this:

Match numbers {
| 'cons.{n _} n
| 'nil 0
}

That expression evaluates to 0 if the numbers list is empty and evaluates to the first number in the list if numbers is not empty.

More recently, I added an alternative way to use pattern matching. It looks like this:

If [Pattern 'cons.{n _} Matches numbers]
    n
    0

That expression does almost the same thing as the previous example.

(It's different in that it only checks for the cons case. Normally, the only other possibility is nil but the code here only distinguishes between cons and not-cons.)

It is equivalent to the following expression:

Match numbers {
| 'cons.{n _} n
| _ 0
}

(The _ is used as a match-anything, bind-no-variables pattern.)

In Language 84, [Pattern 'cons.{n _} Matches numbers] is an expression and can be used anywhere a boolean value is expected.

But as the example shows, if the expression is in a control flow predicate context, it also binds variables appearing in the pattern for use in the branch taken when the condition evaluated to True.

Something interesting happens when you start using this style with And and Or operators.

If (And [Pattern 'sym.t Matches token] (STRING.equal s t))
    [TRUE_BRANCH]
    [FALSE_BRANCH]

In that expression, the t variable is bound both in the (STRING.equal s t) expression and in the [TRUE_BRANCH] of the If conditional.

Translating (naively) to the other pattern matching style, you get the following:

Match token {
| 'sym.t
    If (STRING.equal s t)
        [TRUE_BRANCH]
        [FALSE_BRANCH]
| _
    [FALSE_BRANCH]
}

Note that [FALSE_BRANCH] appears twice. To correctly expand the new style into the old style, I'll need to introduce join points into my intermediate languages to avoid code duplication. I also have to overcome the problem that the naive expansion shown above incorrectly creates a binding for t in the first [FALSE_BRANCH], where 'sym.t matches token but (STRING.equal s t) evaluates to False.

I know that Rust has something similar in its if let construct and I did have that in mind as I worked on this feature even though I haven't used Rust except to write a couple of demo programs to explore its features.

Are you familiar with similar features in a language you use? Is it something that works well? Are there issues to watch out for? I'm curious because I haven't really used a language that works like this so I'm somewhat wary about including it in my own language design.

One thing that worries me is that it feels somehow non-compositional. If you look at the above example that starts If (And ...), you can see that the condition within the If expression affects the translation of the enclosing If expression itself.

For fun, here's a curious function definition I wrote using this new style.

Define (any items pred)
    Iterate items
        (And [Pattern 'cons.{item items} Matches items]
            (Or (pred item) (Continue items)))

20 comments · 91% upvoted · other discussions

comments

ghkbrew

But as the example shows, if the expression is in a control flow predicate context, it also binds variables appearing in the pattern for use in the branch taken when the condition evaluated to True

This is the bit I would expect you and your users to stumble over the most. It's a pretty unique feature as far as I know. The closest feature I can think of is flow typing in some languages, but even that doesn't introduce new bindings.

The scoping of those bound variables seems like it will be complex.

Are they bound in other parts of the conditional expression? Which parts?

What about matches in negative contexts? e.g.

If (Not ['some.x Matches item])
   Print x

Can they be nested under user-defined functions?

Def If_Not cond if_result else_result = 
  If ( Not cond )
    if_result
    else_result

If (If_Not ['some.x Matches item] True x )
   ...
ericbb [OP]

Exactly. These are the kinds of concerns I have too.

The scoping of those bound variables seems like it will be complex.

There is definitely some complexity (that is unique compared to other binding features of the language) but I think it can be made precise:

Each control flow operator that dispatches based on truth values has its own interactions with bindings created by embedded pattern match forms:

If TEST THEN ELSE

Variables bound in TEST are in scope in THEN and not in scope in ELSE.

(And TEST1 TEST2 TEST3 ...)

Variables bound in TEST[m] are in scope in TEST[n] for all n > m.

When one control flow form is nested within another, bindings flow outward (with respect to nesting) based on logical implication, in some sense. (This part is harder to state clearly.)

The Or case has a wrinkle of its own: if there is a pattern matching subexpression, then all subexpressions must be pattern matching expressions that bind the same variables. It's pretty complicated but I haven't run into a use for that case yet. (Mostly, the composed form I find uses for is If (And ...).)

What about matches in negative contexts?

If (Not ['some.x Matches item])
    Print x

I think the answer there has to be that x would not be bound in the Print x expression. If a pattern doesn't match, then there are no bindings to introduce. For what it's worth, the syntax for your example would typically be:

When ![Pattern 'some.x Matches item] {
    (print x)
}

The ! operator is basically just a function and not really a control flow operator in the way that If, And, and Or are. It's like ! versus && and || in C, in case you are familiar with C.

Can they be nested under user-defined functions?

That is a really good question and I think it relates to "custom match pattern" of F# mentioned in another comment here. For now, I have no support for such constructs but it may be possible to find a workable design based on that idea, though I think the "user-defined function" would have to be more of a miniKanren-style "user-defined relation".

Edit: Slight word choice changes (s/bound/in scope/).

ghkbrew

I agree that these are pretty much the only sensible answers to those questions. But at that point you're basically adding a 3rd class of syntax (besides expression and patterns) with it's own set of rules/restrictions and means of abstraction.

I wonder if it's worth the complexity, given that for the most part it's macro-translatable in to match expressions.

What about a slightly different alternative where you just extend the pattern syntax with And, Or, and Not operators. And maybe add a rust-style if let statement.

ericbb [OP]

But at that point you're basically adding a 3rd class of syntax (besides expression and patterns) with it's own set of rules/restrictions and means of abstraction.

Yes. I think that that's the right way to think about it. And, in fact, once you look at it that way, maybe you get compositionality back because now If is defined in terms of composition of two expression branches with one thing of this new class, whose composition rules are not the composition rules of expressions.

I wonder if it's worth the complexity, given that for the most part it's macro-translatable in to match expressions.

Yeah, it could be that it's not worth the complexity. My experience so far is that I do appreciate having these new forms available and I enjoy using them. They seem like fairly natural extensions of the existing concepts in the language. Still, it's important that they carry their weight and don't have nasty corner cases or bad interactions with other features.

What about a slightly different alternative where you just extend the pattern syntax with And, Or, and Not operators. And maybe add a rust-style if let statement.

It's worth a thought but I suspect that going that route might lead back to what I already have. (?)

Edit: To expand on this last point: One reason not to use if let is that Language 84 has a number of different boolean-based conditional operators and I'd like this feature to work with all of them (orthogonally) rather than just with If. Here are the operators (everywhere you see TEST, that's where I'd like to be able to use patterns):

If TEST THEN ELSE

(And TEST1 TEST2 ...)

(Or TEST1 TEST2 ...)

Cond {
| TEST1 BLOCK1
| TEST2 BLOCK2
...
}

When TEST { STATEMENTS }

Begin Cond {
| TEST1 STATEMENTS1
| TEST2 STATEMENTS2
...
}