Function-like Macro

A function-like macro in Rust is a metaprogramming construct invoked using the macro invocation operator (!) that accepts a stream of tokens as input and expands into a new stream of tokens at compile time. Unlike standard functions, which operate on evaluated values at runtime, function-like macros operate on Token Streams (or Token Trees) during the parsing phase, before the full Abstract Syntax Tree (AST) is constructed. This enables syntactic abstraction and arbitrary code generation.

Rust provides two distinct mechanisms for defining function-like macros: Declarative Macros and Procedural Macros.

Declarative Macros (macro_rules!)

Declarative macros are defined using the built-in macro_rules! construct. They operate via a pattern-matching engine that evaluates the input token stream against a series of defined rules. When a match is found, the macro expands according to the corresponding transcriber block.

Syntax Visualization:

#[macro_export]
macro_rules! macro_name {
    ( $metavariable:fragment_specifier ) => {
        // Transcriber (expansion code)
    };
    // Additional rules...
}

• Metavariables: Prefixed with $, these capture matched tokens from the input stream.
• Fragment Specifiers: Define the expected syntax node type for the metavariable (e.g., expr for expressions, ident for identifiers, ty for types, tt for token trees).
• Repetitions: Indicated by syntax like $( ... ),*, allowing the macro to accept and expand variable-length lists of tokens.

Structural Example:

macro_rules! parse_elements {
    ( $( $val:expr ),* ) => {
        {
            let mut temp_vec = Vec::new();
            $(
                temp_vec.push($val);
            )*
            temp_vec
        }
    };
}

Macro Hygiene

A fundamental feature that distinguishes Rust’s declarative macros from C-style text replacement is macro hygiene. Declarative macros are partially hygienic. This means that local variables, labels, and other identifiers declared within the macro’s transcriber block (such as temp_vec in the example above) are resolved in the macro’s definition context, not the caller’s context. This protects the macro’s internal variables from accidentally clashing with variables in the scope where the macro is invoked.

Note: Hygiene in declarative macros applies primarily to local variables and labels. Items like functions or types are resolved in the call site’s context. Procedural macros, by contrast, are unhygienic by default, requiring manual hygiene management using the Span API.

Scoping and Visibility

Because declarative macros operate differently from standard functions, they do not use the pub keyword for visibility. Instead, declarative macros use the #[macro_export] attribute to be visible outside their defining module. Applying #[macro_export] hoists the macro into the root scope of the crate, making it accessible to external crates. Without this attribute, a declarative macro is only visible within the module where it is defined and its submodules.

Procedural Function-like Macros

Procedural function-like macros are defined as public functions within a specialized crate type (proc-macro = true). They are written in standard Rust and execute as compiler plugins at compile time to transform an input TokenStream into an output TokenStream.

Syntax Visualization:

use proc_macro::TokenStream;

#[proc_macro]
pub fn macro_name(input: TokenStream) -> TokenStream {
    // 1. Parse the input TokenStream (often using the `syn` crate)
    // 2. Manipulate the parsed syntax tree
    // 3. Serialize the modified syntax tree back into a TokenStream (often using `quote`)
    
    TokenStream::new() // Returns an empty TokenStream to satisfy the type signature
}

Unlike declarative macros, procedural macros do not rely on declarative pattern matching. Instead, they receive the raw TokenStream, allowing for arbitrary programmatic inspection and manipulation before emitting the final TokenStream.

Invocation Syntax

Regardless of whether a function-like macro is declarative or procedural, it is invoked by appending a bang (!) to its identifier. The input token stream can be enclosed in parentheses (), brackets [], or braces {}. The compiler treats these delimiters identically for the purpose of macro expansion, though convention dictates their usage based on the macro’s structural output.

// Parentheses (typically used for expression-like expansions)
my_macro!(arg1, arg2);

// Brackets (typically used for array/vector-like expansions)
my_macro![arg1, arg2];

// Braces (typically used for block-like expansions)
// Note: Semicolons are not required after brace invocations when used as either items or statements.
my_macro! {
    arg1, arg2
}

Expansion Mechanics

1. Lexing: The compiler converts the source code into a stream of raw tokens.
2. Token Tree Parsing: The compiler groups these raw tokens into token trees (tt) by matching delimiters ((), [], and {}).
3. Invocation Parsing: The compiler encounters the ! operator and identifies the macro invocation.
4. Expansion: The macro consumes the token stream within its delimiters. For declarative macros, the compiler evaluates the matchers sequentially. For procedural macros, the compiler executes the compiled macro binary, passing the tokens as arguments.
5. Integration: The resulting TokenStream is parsed and integrated into the AST, replacing the original macro invocation. The expanded code must form valid Rust syntax for the specific context in which it was invoked (e.g., an expression context, an item context, or a statement context).

Master Rust with Deep Grasping Methodology!

Learn More