Self Modifying Code as an alternative to macros

  • Click bait title!

    The author knows full well, and even acknowledges in the very first sentence, that this article is not about self modifying code as commonly understood. This article is about generated source code.

  • Before CI became prevalent, I was against checked-in code generation and it took me a while to unprogram that bias and re-evaluate. Its been a big help in keeping runtimes fast with a python program at a previous job and at keeping build-times fast in an open source project of mine [0]

    I wish there was a simple way in Rust to make proc macros optionally generate code that gets checked in, allowing for most of the macro costs to be shifted to dev-dependencies. While some macros might be generating code for very dynamic parts of applications, others like clap (CLI parser) are less likely to change and could benefit from it.

    [0] https://epage.github.io/blog/2019/10/speeding-up-rust-builds...

  • > Alas, counting in macro by example is possible, but not trivial.

    It's not stabilized yet, but RFC3086[1] introduces macro metavariable expressions, which make counting trivial!

    The example macro becomes:

        macro_rules! define_error {
          ($($err:ident,)*) => {
            #[derive(Debug, Clone, Copy, PartialEq, Eq)]
            pub enum Error {
              $($err,)*
            }
        
            impl Error {
              pub fn from_code(code: u32) -> Option<Error> {
                match code {
                  $( ${index()} => Some(Error::$err), )*
                  _ => None,
                }
              }
            }
          };
        }
    
    You can see this in practice on the playground[2].

    [1]: https://github.com/rust-lang/rfcs/blob/master/text/3086-macr... [2]: https://play.rust-lang.org/?version=nightly&mode=debug&editi...

  • Generating code is a powerful tool that I think everyone should have in their toolbox — use it wisely of course, but don’t avoid it on principle.

    This describes an interesting approach that I hadn’t seen. I personally wouldn’t have considered putting the result in the same file because I usually like to gitignore any and all generated code, mostly to reduce phantom merge conflicts. However, some colleagues have the opinion that those conflicts are trivially resolved by rebuilding and thus don’t matter, and having diffs of the generated code show up in pull requests adds value.

    What do others think, check in generated code or put it in gitignore?

  • Now that rust-analyzer has "expand macro recursively", you actually can see what a macro expands into fairly easily, so I don't think the unreadability of macros is much of a concern anymore these days. A bigger problem is the error messages you get when you write invalid code inside one.

  • matklad's writings are such a gem!

    I think this is a great investigation into a problem that's somewhat self-inflicted by rust, kind of like how so many of those GoF "design patterns" were simply a response to the lack of expressivity of Java.

    For example, I think this problem doesn't even really make sense in Zig, where types are simply runtime values, and you have the full language available to you in a "compile time" pass. That approach is not without issues of its own (e.g. the article's point about IDE insight into the code, but writ even larger), but it's neat to see a different exploration in the design space, and how certain choices can make some issues just melt away.

    Disclaimer: I'm both a rust and zig newb, and while I think I know roughly how I'd do this on the Zig side, I might be missing something.

  • I would probably manually update the enum and the corresponding match statement. If possible, I'd write a test to check that the two don't get out of sync.

    I'm not too familiar with Rust so I'm not sure if the test code would be simpler than the macro solutions or the hack, but I think it would be.

  • I think Walter got it spot on by not including any type of macro in the D language.

    Personally waiting for his version of "Goto Statement Considered Harmful" but for the macros.

  • undefined

  • I am not familiar enough with Rust to understand what is going on here -- is this like Ruby's never-closed classes and metaprogramming?

  • No virus checker likes self modifying code

  • Is it just me that doesn't see a problem with the manual approach at all? Surely, there may be a better way to keep this in sync, but this is not very high effort.

    Unrelated, but I can't help but think this should use try_from or even potentially panic. If you get an error type you don't recognize, that could pose a risk.

    At least with the manual approach you can decide if particular ranges are safe to ignore or not and do something different. Granted, I'm dreaming up scenarios that are very likely outside the author's intention, so I don't think my thoughts on the matter necessarily refute anything being said.