The visitor pattern is a technique for dynamic dispatch on two values (typically one represents 'which variant of data are we working with' and the other 'which operation are we performing'). You would not generally use that in recursive descent parsing, because when parsing you don't have an AST yet, so 'which variant of data' doesn't make sense, you are just consuming tokens from a stream.
My guy... Do you think that parsers just like... concat tokens into tuples or something....??? Do you not understand that after lexing you have tokens (which are a "type") and AST node construction (an "operation") and that the grammar of a language is naturally a graph.... Like where else would you get the "recursion" from....
If that doesn't make sense I invite you to read some literature:
> makeAST():
> asks the tokenizer for the next token t, and then asks t to call the appropriate factory method the int token and the id token call makeLeaf(), the left parenthesis token calls makeBinOp() all other tokens should flag an error! does the above "smell" like the visitor pattern to you or not? Who are the hosts and who are the visitors?
as an adendum, the reason I don't understand the motivation is that the visitor pattern in the way I described it is useful when you have many different operations to perform on your AST. If you have only one operation on tokens - parsing into an AST - I'm not sure why you need dynamic dispatch on a second thing, the first thing being the token type. Maybe the construction is that different operations correspond to different 'grammar rules'?
I'll repeat myself: this is not possible because you need to recursively construct the nodes (how else would you get a tree...).
You're overindexing on maximally generic visitor pattern. If you have one type of visitor but nonetheless dispatch based on type that's still visitor pattern.
EDIT: to be honest who even cares. My initial point was why in the hell would you stop reading a book because a particular "pattern" offends you. And I'll reassert it here: who cares whether a recursive descent parser fits the exact definition of visitor pattern or not - you have members of a class that do stuff (construct AST nodes) and possibly track other data and then call other members. I usually call that a visitor class even if it's the only one that ever exists <shrug>
I did continue reading the book (not the original author of that reply) but I do think it is distracting for newbies. I had to come back to this page over and over again to recollect memory about the pattern, because I usually read it one chapter or a few sections every week, so every time I had to remind myself how this visitBlah() and accept() pair works. I really think a big switch() (or anything that works but is simpler) would be a lot easier to understand.
The other reason I dislike this kind of stuffs is that I have someone in the team who really likes to use patterns for every piece of code. It's kinda difficult to tell whether it is over-engineering or not, but my principle is that intuition always beats less lines of code (or DRY), unless it is absurdly more lines of code or repetition. And to test that principle you just grab a newbie and see which one makes more sense to him.
It's much easier conceptually to implement this using recursion instead of a while loop and a token stack (it's basically DFS). So I disagree with you there.
> The other reason I dislike this kind of stuffs is that I have someone in the team who really likes to use patterns for every piece of code. It's kinda difficult to tell whether it is over-engineering or not, but my principle is that intuition always beats less lines of code (or DRY), unless it is absurdly more lines of code or repetition. And to test that principle you just grab a newbie and see which one makes more sense to him
I'm with you - I really don't give a shit about patterns (which was my whole original point - who cares). But that last part I don't agree with - systems code (like a parser) doesn't need to be legible to a noob. Of course we're talking about a textbook so your probably right but like I said most production parsers and AST traversals are written exactly this same way. So anyone learning this stuff hoping to get a job doing it should just get used to it.
Visitor thoroughly confuses me in the context of parsing (maybe in all contexts.)
visit and accept are not the verbs I want to be seeing in the code. I want to see then, or, and try.
Ie the link I posted above
This is just and alternative implementation of the visitor pattern. Whether you implement it using dynamic dispatch or a switch or an if stack its all the same pattern…
The visitor pattern is not something I find simple and easy to approach
...or just a big old, plain jane switch statement.
In my current project I modified my ASDL generator to output a C instead of C++ AST and the visitor pattern carried over until realizing a switch statement is just as good (or better) in C so I ripped out that part of the template file. The choice was to write a dispatch function which called the various methods based on the AST node type or have a generated struct full of function pointers with a generated dispatch function which calls the various methods based on the AST node type. Same difference, really, just one has an added level of indirection.
The amazing part is I didn't rewrite the ASDL generator for the fifth time and just decided it's 'good enough' for what I need it for. Aside from one small C++ism, which is easily worked around and turns out wasn't even needed in the C++ template, the thing is 100% language and 'access pattern' agnostic in generating the output code.
There was probably a point I was trying to make when I started typing, dunno?
- You don't need a visitor pattern if you have predetermined the data you are going to work with and all the operations on it (i.e., the open/closed principle does not apply.)
- For the same reason, you don't need dynamic dispatch, which is often how the visitor (and other) pattern(s) are implemented.
- The code is much simpler to understand (and debug) because it's all there in once place. It's also faster than the dynamic dispatch version because it's all known at compile-time.
- Personally: OOP is stupid, confusing, and inefficient; I think universities should only teach it as an optional course. These patterns are 50% lack of functional programming features and 50% sheer stupidity. Universities should go back to teaching real abstraction with Scheme and SICP, a MIPS-style assembly language, and stop confusing students.
The Next Big Trick™ is to just embed the function pointer into the opcode itself and do away with the dispatching completely, getting rid of a single pointer dereference per opcode has to be worth at least a 0.01% speed gain, right? I'm kidding, of course, as the original copy and patch (using C labels as references to mark the code boundaries of the code templates) should allow actual measurable gains in the single digit range.
Statement Data Classes: https://github.com/alabhyajindal/plox/blob/main/stmt.py
If statements in the parser matching against them: https://github.com/alabhyajindal/plox/blob/main/parser.py#L3...
I preferred my simple C design whic used a lookup table to quickly access a structure for each state, event pair instead. The structure would provide an output state, a message to send, an action to emit, a bitmask to select zero or more other standard operations, and yes a function to run in those rare cases where something really special was needed. All fields optional.
The point is I didn't need to write N x M functions, I just needed to edit a table. And I didn't need to understand any rocket science.
> Abdulaziz Ghuloum, 2006, An Incremental Approach to Compiler Construction http://scheme2006.cs.uchicago.edu/11-ghuloum.pdf
[0] https://mitpress.mit.edu/9780262047760/essentials-of-compila...
The author also maintains a list of implementations created from the book: https://github.com/nlsandler/c-compiler-implementations
Scala is an awesome language which frees one from working on many boring details and makes it possible to keep the codebase tiny. With such an expressive language I can concentrate on the logic instead of thinking about minor things.
We have enough memory and cpu power to use worse than linear algorithms without noticeable performance impact.
Parsers aren't an issue at all in our days, peg combinators like fastparse allow one to be extremely productive.
I tend to stick to immutable multi-staged pipelile with several immutable trees, use error-accumulating data structures (Either[NonEmptyList[Issue], T]), explicitly express entity (eg type definition) dependencies as graphs (which can be processed iteratively and in parallel).
mkw5053•6mo ago