Query Expression

A query expression is a declarative, SQL-like syntax in C# used to filter, project, and manipulate data across enumerable collections. It acts as syntactic sugar for Language Integrated Query (LINQ) method syntax, allowing developers to write complex queries using language-level keywords. At compile time, the C# compiler translates these query expressions into standard extension method calls (such as Where, Select, and OrderBy) and lambda expressions.

Syntax Anatomy

A query expression must strictly begin with a from clause and terminate with either a select or group clause. Between the initial from and the terminal clause, intermediate query body clauses can be interleaved in almost any order and repeated multiple times.

from [type] [range_variable] in [data_source]
// Intermediate clauses can be interleaved and repeated in any order:
// [from_clause] | [let_clause] | [where_clause] | [join_clause] | [orderby_clause]
[select_clause | group_clause]
// Optional query continuation:
// [into_clause]

Core Mechanics

• Range Variables: The from clause introduces a range variable, which represents each successive element in the data source. The compiler infers the type of the range variable based on the IEnumerable<T> or IQueryable<T> implementation of the source.
• Deferred Execution: Defining a query expression does not execute it. The expression merely constructs an expression tree or an enumerable state machine. Execution is deferred until the query is iterated over (e.g., via a foreach loop or a materialization method like ToList()).
• Projection: The terminal select clause dictates the shape and type of the returned data. It can return the element as-is, extract a specific property, or project the data into a new object or anonymous type.

Clause Specifications

• from: Initializes the query, specifying the data source and the range variable. Multiple from clauses can be chained to perform cross-joins (Cartesian products) or to query nested collections.
• where: Applies a boolean predicate to filter the sequence. Elements that evaluate to false are discarded from the pipeline.
• select: Defines the final projection of the sequence.
• group ... by: Aggregates elements into an IGrouping<TKey, TElement> sequence based on a specified key extraction expression.
• orderby: Sorts the sequence based on one or more keys. It supports ascending (default) and descending modifiers.
• join: Correlates elements from two distinct data sources based on matching keys. It requires the equals keyword for the equijoin condition.
• let: Computes a value and binds it to a new range variable, preventing the need to recalculate the same expression multiple times in subsequent clauses.
• into (Query Continuation): When used after a select or group clause, into performs a query continuation. It binds the result to a new range variable and creates a new query. All previous range variables go out of scope.
• into (Group Join): When used after a join clause, into performs a group join. It groups the correlated inner elements into an IEnumerable<T> and binds them to a new range variable, while keeping the outer range variable in scope.

Compiler Translation (Lowering)

The C# compiler mechanically translates query expressions into method invocations based on the Query Expression Pattern. The target types do not need to implement a specific interface; they only need to expose methods with the correct signatures (often achieved via extension methods).

Query Expression Syntax:

IEnumerable<string> query = 
    from user in users
    where user.Age > 18
    orderby user.Name descending
    select user.Name;

Compiled Method Syntax:

IEnumerable<string> query = users
    .Where(user => user.Age > 18)
    .OrderByDescending(user => user.Name)
    .Select(user => user.Name);

Transparent Identifiers

When a query expression contains let clauses or multiple from clauses, the compiler generates anonymous types to carry multiple range variables through the query pipeline. These are known as transparent identifiers. They are an internal compiler mechanism to maintain scope and state across the translated method chain without requiring the developer to manually construct complex tuple or anonymous type projections at each step.

Master C# with Deep Grasping Methodology!

Learn More