You are an expert on programming language design and meta-programming. Design a…
- User
You are an expert on programming language design and meta-programming. Design a programming language that is useful to design a programming languages. It should have:
- an EBNF grammars for every syntactic constructs
- a go-like scripting language that we can use to do meta-programming and introspection
- multiple DSL language descriptions used to generate parsers, and specifically its own parser, pretty-printer and evaluator
- a OOP flavor
- a single file source code You will give a comprehensive plan and after that fully describe the multiple sections that comprise the programming language. After the plan and each sections prompt me for review.
Assistant
This is an exciting challenge! Designing a language for language design is a powerful concept, focusing on reflection, generation, and extensibility.
I propose a language named GrammarForge. Its core philosophy is to provide a highly reflective, self-describing, and generative environment for defining, parsing, transforming, and evaluating programming languages.
Comprehensive Plan: GrammarForge
1. Vision & Core Philosophy: GrammarForge aims to be the ultimate meta-language. It's designed from the ground up to facilitate the rapid prototyping and robust implementation of programming languages, DSLs, and compilers/interpreters. Its key principle is "bootstrapping via self-description," meaning GrammarForge will define its own syntax, AST structure, pretty-printing, and evaluation rules using its internal DSLs. This makes the language inherently extensible and understandable.
2. Key Components:
- Grammar Definition Language (GDL): An EBNF-inspired DSL for formally defining the concrete syntax of languages. It provides constructs for sequences, alternatives, repetitions, and optional elements, tightly integrated with AST node creation.
- Abstract Syntax Tree (AST) Definition Language (ADL): A declarative DSL for defining the structure of AST nodes. It specifies fields, their types, and relationships (e.g., parent-child, lists). ADL will often be implicitly derived from GDL rules but allows for explicit refinement. It will embrace an OOP flavor, allowing for inheritance and polymorphism among AST node types.
- Semantic Action & Transformation Language (SATL): This is the imperative, Go-like scripting language. It serves as the "glue" and "action" layer. SATL is used for:
- Defining semantic actions during parsing (e.g., constructing AST nodes from recognized grammar rules).
- Implementing sophisticated AST transformations (optimization, refactoring).
- Defining pretty-printing logic that goes beyond declarative rules.
- Implementing the core evaluation/interpretation logic for ASTs.
- General meta-programming, introspection, and querying the GDL/ADL definitions.
- Pretty-Printer Description Language (PDL): A declarative DSL for mapping AST nodes back to their textual representation. It specifies formatting rules, indentation, line breaks, and spacing, with hooks for SATL for complex formatting needs.
- Evaluator Description API (within SATL): Instead of a separate DSL, evaluation logic will be defined through a rich API within SATL. This allows for the full power of the Go-like scripting language to be applied to the semantics of any language defined in GrammarForge. It provides mechanisms for dispatching evaluation logic based on AST node types, managing execution context, and handling values.
3. Self-Hosting & Single File Source:
- Self-Definition: GrammarForge will use its own GDL to define its syntax, its own ADL to define its AST structure, its own SATL for its parser actions and initial evaluator, and its own PDL for its pretty-printer. This is the cornerstone of its design, demonstrating its capabilities and providing a complete, runnable example of language definition within GrammarForge.
- Single File Structure: The entire definition of GrammarForge itself (its GDL, ADL, PDL, and initial SATL logic) will reside in a single
.gffile. A minimal "bootstrap" runtime (potentially written in Go, Rust, or C for performance and simplicity) will be responsible for loading, parsing, and initializing this single.gffile, thereby bringing the full GrammarForge system to life. This emphasizes portability and ease of distribution.
4. OOP Flavor:
- AST Nodes as Objects: All AST nodes generated or manipulated within GrammarForge are treated as objects with properties (fields) and potentially methods.
- SATL's OOP Support: SATL will feature object-oriented constructs, including structured types (similar to Go structs but with methods), method invocation, and interfaces for polymorphic operations, particularly useful when working with diverse AST node types.
- Inheritance/Composition in ADL: The ADL will support defining relationships between AST node types that resemble inheritance (e.g., an
Expressionnode might be a base type forBinaryOp,Literal,VariableRef).
5. Development Workflow (Conceptual):
- Define the target language's syntax using GDL.
- Refine the target language's AST structure using ADL (often implicitly derived from GDL).
- Implement parsing logic (semantic actions) in SATL to construct the AST from recognized tokens.
- Implement pretty-printing rules using PDL, with SATL for advanced formatting.
- Implement the evaluator/interpreter for the AST using the SATL's evaluation API.
- Optionally, write SATL scripts for static analysis, transformation, or code generation for the target language.
- Use GrammarForge's
buildcommand to generate a stand-alone parser, pretty-printer, or interpreter for the target language.
I've outlined the comprehensive plan. Please let me know if you'd like to review this plan before I proceed with describing each section in detail.
Prompt for Review: Please review the comprehensive plan.
- User
Please proceed.
Assistant
Excellent! Let's dive into the detailed description of each component of GrammarForge.
Section 1: Grammar Definition Language (GDL)
The Grammar Definition Language (GDL) is the cornerstone of GrammarForge, providing a formal, EBNF-inspired syntax for defining the concrete syntax of any language, including GrammarForge itself. It's designed to be intuitive for anyone familiar with context-free grammars while directly enabling the construction of Abstract Syntax Trees (ASTs).
Purpose:
- To define the terminal and non-terminal symbols of a language.
- To specify the production rules that dictate how symbols combine.
- To implicitly or explicitly guide the generation of AST nodes during parsing.
- To serve as the input for a parser generator within GrammarForge.
Syntax & Concepts:
GDL uses a syntax that blends EBNF conventions with a more structured approach to associate grammar rules directly with AST node types and fields.
1. Rule Definition:
A rule defines a non-terminal. It begins with the rule name, followed by
::=, and then the production body.RuleName ::= ProductionBody2. Production Body Elements:
- Literals (Tokens): Enclosed in single or double quotes.
'keyword'"operator"
- Non-terminals: Referenced by their rule name.
AnotherRule
- Sequencing: Elements placed next to each other indicate a sequence.
RuleA RuleB RuleC
- Alternatives: Separated by
|.OptionA | OptionB | OptionC
- Optional Elements: Enclosed in
[ ].[OptionalPart](0 or 1 occurrence)
- Repetition (Zero or More): Enclosed in
{ }.{RepeatedPart}(0 or more occurrences)
- Repetition (One or More): Enclosed in
( )+.(RequiredPart)+(1 or more occurrences)
- Grouping: Enclosed in
( )for precedence or local application of operators.(PartA | PartB)
3. AST Node Association (Implicit & Explicit):
GDL is tightly coupled with AST generation. By default, each GDL rule generates an AST node of the same name. Elements within the rule can become fields of that AST node.
-
Implicit Field Naming: When a non-terminal or a repeated/optional element is part of a sequence, it automatically becomes a field of the containing AST node, named after the rule or a derived name.
// GDL FunctionDecl ::= 'func' Identifier Parameters Body ; // Implied AST Node (ADL-like) type FunctionDecl struct { Name Identifier Params Parameters Body Body } -
Explicit Field Naming (
:operator): Use:to explicitly name a field, especially when the element's name isn't suitable or when multiple instances of the same non-terminal appear.// GDL BinaryOp ::= left:Expression operator:('+' | '-' | '*' | '/') right:Expression ; // Implied AST Node (ADL-like) type BinaryOp struct { Left Expression Operator string // Or a specific Token type Right Expression }Note: Tokens captured with an explicit name will often become
stringfields or a dedicatedTokentype, capturing their literal value. -
List Fields: Repetitions (
{ }or( )+) naturally become lists (slices/arrays) of their contained type.// GDL ParameterList ::= '(' { Parameter ',' }? ')' ; // Parameter followed by optional comma // Implied AST Node (ADL-like) type ParameterList struct { Params []Parameter // Or a list of Parameter nodes directly }Note: Common patterns like
A (',' A)*for comma-separated lists are automatically recognized as lists. -
Choice/Alternative Nodes (
@union): When a rule has alternatives, it implicitly suggests a polymorphic AST node, where the actual type is one of the alternatives. This maps directly to the OOP features of ADL.// GDL Expression ::= Literal | BinaryOp | CallExpr | VariableRef ; // Implied AST Node (ADL-like) type Expression interface { /* common methods for expressions */ } // Or a union type in languages that support it. // In GrammarForge's ADL, it would be explicitly defined as: // @union Expression = Literal | BinaryOp | CallExpr | VariableRef;
4. Lexical Rules (Tokens):
GDL rules can also define terminal symbols (tokens) using regular expressions. These are typically defined first.
Identifier ::= `[a-zA-Z_][a-zA-Z0-9_]*` ; IntegerLit ::= `[0-9]+` ; StringLit ::= `\"([^"\\]|\\.)*\"` ; Whitespace ::= `\s+` -> skip ; // Special directive to ignore whitespace Comment ::= `//.*` | `/*(\*[^/]|[^*])*\*/` -> skip ; // Multi-line and single-line comments-> skipdirective indicates that this token should be recognized but discarded, not appearing in the AST.- Token precedence can be implicitly determined by order of definition or explicitly specified (though for simplicity, explicit precedence rules are often handled in the parser generator or semantic actions).
5. GDL Directives:
GDL can include special directives for the parser generator or for linking with SATL.
%start RuleName;: Specifies the top-level starting rule for the grammar.%import "other_grammar.gf";: Allows importing rules from other GDL files, facilitating modularity.
Example (Fragment of GrammarForge's own GDL):
// --- Lexical Rules for GrammarForge GDL --- Identifier ::= `[a-zA-Z_][a-zA-Z0-9_]*` ; StringLiteral ::= `\"([^"\\]|\\.)*\"` ; RegexLiteral ::= ```([^`\\]|\\.)*` ; Whitespace ::= `\s+` -> skip ; Comment ::= `//.*` | `/*(\*[^/]|[^*])*\*/` -> skip ; // --- GDL Grammar Rules --- %start GrammarFile; GrammarFile ::= { Directive | RuleDecl } ; Directive ::= '%' ( StartDirective | ImportDirective ) ';' ; StartDirective ::= 'start' ruleName:Identifier ; ImportDirective ::= 'import' path:StringLiteral ; RuleDecl ::= ruleName:Identifier '::=' ruleBody:RuleBody ';' ; RuleBody ::= ( term:RuleTerm { op:('|' | RuleTerm) term:RuleTerm } ) ; RuleTerm ::= element:RuleElement [ ( '?' | '*' | '+' | '{' count:IntegerLiteral '}' ) // Optional, zero-or-more, one-or-more, or exact count ] ; RuleElement ::= name:Identifier ':' element:RuleElement // Named field | literal:StringLiteral | regex:RegexLiteral | ruleRef:Identifier | '(' group:RuleBody ')' ;Integration with other components:
- ADL: GDL rules are the primary input for ADL. GrammarForge's internal tooling derives initial ADL types from GDL rules. The explicit naming (
name:element) directly informs ADL field names. Polymorphic rules (A | B | C) directly translate to ADL interfaces or union types. - SATL: Semantic actions, written in SATL, are associated with GDL rules. When a rule is successfully parsed, the corresponding SATL action is executed, typically to construct the appropriate AST node using values captured from the rule's elements.
- Parser Generator: The GDL definition is directly consumed by GrammarForge's internal parser generator to create a parser for the specified language.
This structured approach makes GDL a powerful tool for defining syntax, inherently guiding the construction of the Abstract Syntax Tree, which is the next crucial step.
Prompt for Review: I've detailed the Grammar Definition Language (GDL). Please review this section before I move on to the Abstract Syntax Tree Definition Language (ADL).
- User
Please proceed.
Assistant
Alright, let's detail the Abstract Syntax Tree Definition Language (ADL).
Section 2: Abstract Syntax Tree Definition Language (ADL)
The Abstract Syntax Tree Definition Language (ADL) is a declarative DSL for precisely defining the structure and relationships of AST nodes. While the Grammar Definition Language (GDL) implicitly suggests an AST structure, ADL allows for explicit refinement, type declarations, and the definition of polymorphic relationships, providing the strong type system needed for robust meta-programming and language analysis.
Purpose:
- To define the precise structure of AST nodes (fields and their types).
- To establish an explicit type system for ASTs, enabling type-safe manipulation in SATL.
- To model polymorphic relationships between AST nodes using an OOP-flavored approach (interfaces/unions).
- To serve as the canonical source for code generation (e.g., creating Go structs/interfaces internally for GrammarForge's runtime).
- To separate the conceptual structure of the program from its concrete textual representation.
Syntax & Concepts:
ADL's syntax is inspired by Go's struct and interface declarations, making it familiar yet highly specialized for AST definitions.
1. Node Type Definition (
type):The primary construct is the
typedeclaration, which defines a new AST node type, typically mapping to a concept in the language being defined (e.g.,FunctionDecl,BinaryExpr).type NodeTypeName struct { FieldName1 FieldType1 FieldName2 FieldType2 // ... }NodeTypeName: A unique identifier for the AST node type.FieldName: The name of a property/field within the AST node.FieldType: The type of the field. This can be a primitive type (likestring,int,bool,float), a reference to another ADL-definedNodeTypeName, or a list of such types.
2. Primitive Types:
ADL supports standard primitive types that typically map to values extracted directly from tokens or simple computed values:
string: For identifiers, string literals, operator symbols, etc.int: For integer literals.float: For floating-point literals.bool: For boolean values.Token: A special type representing a raw lexical token, including its value, kind, and source position. This is useful for preserving full fidelity of the source.
3. Reference Types:
Fields can reference other ADL-defined AST node types, establishing the hierarchical structure of the AST.
type FunctionCall struct { Callee Expression // Reference to an Expression node Arguments []Expression // List of Expression nodes }4. List/Array Types:
Lists of AST nodes are common (e.g., a list of statements, a list of parameters). ADL uses the
[]syntax similar to Go slices.type BlockStatement struct { Statements []Statement // A list of Statement nodes }5. Polymorphic Node Types (Interfaces / Union Types):
This is a core OOP feature of ADL. Often, a single conceptual "slot" in an AST can hold different kinds of nodes (e.g., an
Expressioncan be aLiteral, aBinaryOp, or aCallExpr). ADL addresses this withinterfacedeclarations, similar to Go interfaces.interface Expression { // No explicit methods here, methods are defined in SATL for AST nodes. // This simply declares a contract that concrete types must satisfy (implicitly). } // Concrete types that "implement" (are assignable to) Expression: type Literal struct { Value string // Could be int, float, string literal } type BinaryOp struct { Left Expression Operator string Right Expression } type CallExpr struct { Func Expression Args []Expression } // And then in other structs: type ReturnStatement struct { Value Expression // This field can hold any type that implements the 'Expression' interface. }In GrammarForge's internal representation, this means
Expressionis a type that can hold any ofLiteral,BinaryOp,CallExpr. When a GDL rule has alternatives likeExpression ::= Literal | BinaryOp | CallExpr;, it would automatically generate an ADL interface forExpressionand ensure thatLiteral,BinaryOp, andCallExprare recognized as its concrete implementations.6. Optional Fields:
While not explicitly represented with a
?in ADL syntax like some IDLs, optionality is usually handled by:- Using
nilfor fields that are references to other AST nodes if they are absent. - Using empty slices (
[]) for optional lists. - The
Tokentype might have anIsValid()method to check presence.
7. Comments:
Single-line comments starting with
//are supported.Relationship with GDL:
- Derivation: The primary way ADL types are initially created is by processing GDL rules. Each GDL rule
RuleName ::= ... ;typically generates a correspondingtype RuleName struct { ... }in ADL. - Field Mapping:
- Explicitly named GDL elements (
fieldName:RuleRef) become fields in the ADL struct. - Unnamed non-terminals in a GDL sequence often become fields named after the non-terminal.
- GDL repetitions (
{...},(...)) map to ADL list types ([]). - GDL alternatives (
A | B | C) map to ADLinterfacetypes, withA,B,Cbeing concrete implementations.
- Explicitly named GDL elements (
- Refinement: ADL is crucial for cases where the desired AST structure differs from the most direct translation of the GDL.
- Flattening: Several GDL rules might semantically represent a single AST node type (e.g.,
Term->Factor;Factor->Literal). In ADL, one might flatten this totype Term struct { Value Literal }. - Enrichment: ADL allows adding computed fields or meta-data fields not directly captured by the grammar (e.g.,
type VarDecl struct { Name string; TypeName string; ResolvedType *TypeDefinition }). - Manual Interfaces: While GDL hints at interfaces, ADL allows explicit definition and grouping of types under an interface, providing better semantic clarity.
- Flattening: Several GDL rules might semantically represent a single AST node type (e.g.,
Relationship with SATL:
- Type System: ADL provides the type system for SATL. All AST nodes manipulated within SATL scripts are instances of ADL-defined types. This enables strong typing, type checking, and auto-completion within the SATL environment.
- Object Creation: During parsing, SATL's semantic actions are responsible for instantiating ADL types and populating their fields based on recognized grammar elements.
- Method Definition: SATL is where methods for ADL-defined types are implemented. For example, an
Evaluate()method for anExpressioninterface would be implemented for each concreteExpressiontype (e.g.,Literal.Evaluate(),BinaryOp.Evaluate()). - Introspection: SATL can introspect the ADL definitions (e.g., iterate through all defined AST node types, list their fields and types) for meta-programming tasks.
Example (Fragment of GrammarForge's own ADL for its GDL grammar):
// --- Core AST Node for a GrammarForge Definition --- // GdlNode is the base interface for all elements defined within a GDL file. interface GdlNode {} type GrammarFile struct { Directives []DirectiveDecl // List of directives at the top Rules []RuleDecl // List of grammar rules } inherits GdlNode; // An explicit way to say it implements GdlNode (optional, often implied) // Directives interface DirectiveDecl {} type StartDirective struct { RuleName string // The name of the start rule } implements DirectiveDecl; type ImportDirective struct { Path string // Path to the imported GDL file } implements DirectiveDecl; // Rule Definitions type RuleDecl struct { Name string // Name of the rule (non-terminal) Body RuleBody // The production body } implements GdlNode; interface RuleBody {} // A sequence of alternatives (e.g., A | B | C) type RuleAlternation struct { Alternatives []RuleSequence // Each alternative is a sequence of terms } implements RuleBody; // A sequence of terms (e.g., A B C) type RuleSequence struct { Terms []RuleTerm } implements RuleBody; interface RuleTerm {} // A single element in a rule (e.g., an identifier, literal, or group) type RuleElement struct { Name string // Optional explicit name (e.g., 'left:Expression') Content RuleElementContent // The actual content of the element Modifier RuleModifier // Optional modifier (e.g., '?', '*', '+') } implements RuleTerm; interface RuleElementContent {} type IdentifierRef struct { Value string // The identifier name (e.g., 'Expression') } implements RuleElementContent; type StringLiteral struct { Value string // The literal string (e.g., "'func'") } implements RuleElementContent; type RegexLiteral struct { Value string // The regex pattern (e.g., "`[a-zA-Z_]*`") } implements RuleElementContent; type GroupedBody struct { Body RuleBody // For ( A | B ) } implements RuleElementContent; type RuleModifier struct { Kind string // E.g., "?", "*", "+", or "{count}" Count int // If Kind is "{count}" }This ADL section formally defines the structure of the internal representation of a language described in GrammarForge. This structured data is then processed and manipulated by the Go-like scripting language, SATL, which is our next component.
Prompt for Review: I've detailed the Abstract Syntax Tree Definition Language (ADL). Please review this section before I move on to the Semantic Action & Transformation Language (SATL).
- User
Please proceed.
Assistant
Excellent! Now for the operational heart of GrammarForge: the Semantic Action & Transformation Language (SATL).
Section 3: Semantic Action & Transformation Language (SATL)
The Semantic Action & Transformation Language (SATL) is the imperative, Go-like scripting language within GrammarForge. It acts as the "glue" that binds the declarative language definitions (GDL, ADL, PDL) to concrete operations. It is the environment for implementing all dynamic logic: parsing actions, AST transformations, introspection, pretty-printing customization, and the core evaluation semantics of any language defined within GrammarForge.
Purpose:
- Semantic Actions: To execute code during parsing, typically to construct AST nodes from the recognized grammar elements.
- AST Manipulation: To traverse, analyze, transform, and optimize Abstract Syntax Trees.
- Meta-programming: To introspect and query the GDL and ADL definitions themselves.
- Pretty-Printing Logic: To provide custom formatting logic for complex AST nodes that go beyond declarative PDL rules.
- Evaluation/Interpretation: To define the runtime semantics of a language by implementing an interpreter that operates on its AST.
- General Scripting: For utility functions, testing, and managing the language development workflow.
Go-like Flavor:
SATL adopts key features from Go to offer a familiar, efficient, and robust scripting environment:
1. Syntax & Control Flow:
- Function Declarations:
func myFunction(arg1 Type1, arg2 Type2) (ReturnType1, ReturnType2) { ... } - Variable Declarations:
var x int = 10;or short declarationx := 10;(type inference). - Control Structures:
if/else,forloops (includingfor rangefor collections),switchstatements. - Semicolon Insertion: Optional semicolons, similar to Go's rules.
- Braces: Required for code blocks
{}.
2. Type System & OOP Flavor:
- Static Typing: All variables have a defined type, ensuring robustness and enabling tooling.
- Structs (ADL Nodes): All AST nodes defined in ADL are represented as structs in SATL. You can instantiate them and access their fields directly:
node := ast.NewFunctionDecl(); node.Name = "myFunc";. - Methods: You can define methods on ADL-defined struct types, enabling behavior to be attached directly to AST nodes. This is crucial for implementing visitors, pretty-printers, and evaluators.
// SATL code for an ADL-defined BinaryOp type func (op *ast.BinaryOp) Evaluate(env *EvalContext) (Value, error) { leftVal, err := op.Left.Evaluate(env) if err != nil { return nil, err } rightVal, err := op.Right.Evaluate(env) if err != nil { return nil, err } // ... logic based on op.Operator and leftVal/rightVal types ... return result, nil } - Interfaces (ADL Interfaces): ADL
interfacetypes translate directly to SATL interfaces, allowing for polymorphism. Functions can accept interface types, and concrete AST nodes implementing that interface can be passed.// SATL code: func PrintExpression(expr ast.Expression) { // expr can be ast.Literal, ast.BinaryOp, etc. // We can call methods defined on the Expression interface: prettyString := expr.PrettyPrint() // Assuming PrettyPrint is an interface method println(prettyString) } - Composition/Embedding: Similar to Go, structs can embed other structs, promoting code reuse, especially for common AST node properties (e.g.,
SourceLocation).
3. Error Handling:
- Multiple Return Values: Functions can return multiple values, typically
(result, error). - Explicit Error Checking: Errors must be explicitly checked, fostering robust code.
value, err := myFunc() if err != nil { return nil, err // Propagate error } // Use value - Built-in
errortype: A standard interface for error values.
4. Standard Library / Built-ins: SATL provides a minimal but useful standard library for common tasks:
- I/O:
println,printf, file reading/writing utilities. - String Manipulation:
strings.Split,strings.Join,strings.Contains,strings.ReplaceAll. - Collections: Operations on lists (
len,append,make). - Type Reflection: Basic type checking (
isType(obj, "MyStruct")).
Meta-programming Capabilities:
SATL's power for language design comes from its deep integration with the language's own definition components.
1. Introspection API (GDL/ADL Schema Access): A global
GF(GrammarForge) object is exposed to SATL, providing read-only access to the parsed GDL and ADL definitions.// Example: Iterate through all defined ADL types func ListAllAstNodes() { println("Defined AST Nodes:") for _, node := range GF.Schema.ADL.NodeTypes { println(fmt.Sprintf("- %s (is_interface: %t)", node.Name, node.IsInterface)) if !node.IsInterface { for _, field := range node.Fields { println(fmt.Sprintf(" - %s: %s (is_list: %t)", field.Name, field.Type, field.IsList)) } } } println("\nDefined GDL Rules:") for _, rule := range GF.Schema.GDL.Rules { println(fmt.Sprintf("- %s", rule.Name)) // Access rule body, terms, etc. for more detailed introspection } }2. AST Construction & Manipulation (Parser Actions): When defining a language's GDL, each rule can be associated with a SATL function (a "semantic action"). This function receives the recognized elements of the rule and is responsible for constructing the corresponding ADL-defined AST node.
// GDL rule definition for a function declaration FunctionDecl ::= 'func' Identifier:name Parameters:params Body:body ;// SATL semantic action for FunctionDecl // This function is invoked by the parser when 'FunctionDecl' rule matches. // The arguments 'name', 'params', 'body' are automatically extracted // from the GDL rule's named elements. func CreateFunctionDecl(name ast.Identifier, params ast.ParameterList, body ast.BlockStatement) ast.FunctionDecl { decl := ast.NewFunctionDecl() // Factory function or direct struct literal decl.Name = name decl.Parameters = params decl.Body = body return decl }Note: The binding between GDL rule names and SATL function names for semantic actions is implicit by convention (e.g.,
Create[RuleName]) or explicitly declared in the GDL/a separate mapping file.3. AST Traversal & Transformation: SATL is used to implement visitors or recursive functions for processing ASTs. This enables:
- Static Analysis: Type checking, linting, semantic validation.
- Optimization: Constant folding, dead code elimination.
- Code Generation: Emitting bytecode, C code, or another language.
- Refactoring: Modifying the AST structure.
// Example: Simple AST visitor pattern for an Expression type AstVisitor struct { // ... fields for context ... } func (v *AstVisitor) Visit(node ast.Node) { switch n := node.(type) { // Type assertion for polymorphism case *ast.BinaryOp: v.Visit(n.Left) v.Visit(n.Right) // Perform action for BinaryOp case *ast.Literal: // Perform action for Literal case ast.Expression: // Using an interface type // Special logic for all expressions default: // Handle unknown node types or recurse on children if generic } }4. Pretty-Printing Hooks: While PDL provides declarative pretty-printing, SATL allows for highly customized formatting logic where the declarative rules aren't sufficient (e.g., complex line wrapping, conditional formatting based on node content). SATL functions can be associated with PDL rules as hooks to render specific parts of an AST node.
// SATL pretty-printing helper for a complex list formatting func FormatParameterList(list *ast.ParameterList) string { if len(list.Params) == 0 { return "()" } if len(list.Params) > 3 { // Special formatting for many parameters // Example: Line break after each param paramsStr := []string{} for _, p := range list.Params { paramsStr = append(paramsStr, p.PrettyPrint()) // Call PrettyPrint on sub-nodes } return fmt.Sprintf("(\n %s\n)", strings.Join(paramsStr, ",\n ")) } // Default: inline paramsStr := []string{} for _, p := range list.Params { paramsStr = append(paramsStr, p.PrettyPrint()) } return fmt.Sprintf("(%s)", strings.Join(paramsStr, ", ")) }5. Evaluation API: SATL provides an
EvalContextandValueAPI to build interpreters. TheEvalContextmanages scope, variables, and runtime state. AST nodes typically have anEvaluatemethod (defined in SATL) that dispatches to specific logic based on their type.// SATL code for an interpreter type EvalContext struct { Scope map[string]Value // ... other runtime state ... } interface Value { // Methods common to all runtime values (e.g., Type(), String()) } type IntValue struct { Val int } implements Value; type StringValue struct { Val string } implements Value; // ... other runtime value types ... func (expr *ast.Literal) Evaluate(ctx *EvalContext) (Value, error) { // Assuming Literal.Value is a string, parse it to appropriate SATL Value if isInt(expr.Value) { return &IntValue{parseInt(expr.Value)}, nil } if isString(expr.Value) { return &StringValue{parseString(expr.Value)}, nil } return nil, fmt.Errorf("unknown literal type: %s", expr.Value) } func (stmt *ast.ReturnStatement) Evaluate(ctx *EvalContext) (Value, error) { val, err := stmt.Value.Evaluate(ctx) if err != nil { return nil, err } // How return values are handled depends on language semantics (e.g., panic/recover, special error type) // For now, let's assume a special sentinel value for return. return &ReturnValue{Value: val}, nil }Single File Source Code Implication:
For GrammarForge's own definition, the SATL code that implements its parser, pretty-printer, and initial evaluator will be embedded directly within the single
.gffile, likely within a dedicated// @satlblock or similar, alongside its GDL, ADL, and PDL definitions. This highlights the self-describing nature of the language.This comprehensive Go-like scripting capability makes SATL a powerful engine for bringing defined languages to life, from syntax to semantics.
Prompt for Review: I've detailed the Semantic Action & Transformation Language (SATL). Please review this section before I move on to the Pretty-Printer Description Language (PDL).
- User
Please proceed.
Assistant
Right, let's establish the rules for making our defined languages look good on paper: the Pretty-Printer Description Language (PDL).
Section 4: Pretty-Printer Description Language (PDL)
The Pretty-Printer Description Language (PDL) is a declarative DSL within GrammarForge dedicated to defining how Abstract Syntax Tree (AST) nodes are translated back into well-formatted, human-readable source code. It allows language designers to specify layout, indentation, spacing, and line-breaking rules, ensuring a consistent and idiomatic representation of the language.
Purpose:
- To generate source code from an AST, useful for code generation, refactoring tools, and language servers.
- To enforce a consistent coding style for any language defined within GrammarForge.
- To provide a declarative way to map AST structure back to textual representation.
- To integrate with SATL for complex or conditional formatting requirements.
Syntax & Concepts:
PDL rules are defined per AST node type, specifying how that node and its children should be rendered.
1. Print Rule Definition:
A print rule begins with
printfollowed by the AST node type name (as defined in ADL), and then a block{}containing the formatting instructions.print AstNodeTypeName { // Formatting instructions }2. Basic Elements:
- Literals: Static strings or keywords to be printed. Enclosed in single or double quotes.
'func'"interface"
- Field References: To print the value of an AST node's field. Preceded by
$and followed by the field name (as defined in ADL). If the field is itself an AST node, its corresponding PDL rule will be invoked recursively.$Name$Parameters$Body
- Spacing Directives:
space: Inserts a single space.soft_space: Inserts a space if needed (e.g., avoids double spaces, respects line breaks). This is the default for spaces between elements if not otherwise specified.no_space: Explicitly ensures no space is inserted.
- Line Breaking Directives:
newline: Inserts a mandatory newline.soft_newline: Inserts a newline if needed (e.g., line is too long, or previous element was a block).
- Indentation Directives:
indent: Increases the current indentation level for subsequent elements.dedent: Decreases the current indentation level.block: A convenience directive that typically impliesnewline,indent, then the content, thendedent, thennewline. Useful for code blocks.
3. List Formatting (
list):For fields that are lists (arrays/slices) of AST nodes (as defined in ADL), the
listdirective provides powerful formatting options.list $FieldName [separator StringLiteral] // Optional separator string (e.g., ', ') [each { ... }] // Block applied to each element [join_with soft_newline] // Or newline, space, soft_space [indented] // Apply indentation to each item [wrap_after_count IntegerLiteral] // Break line after N items { // Content to print for each element. Use $element to refer to the current list item. $element }$element: Special placeholder referring to the current item being processed in the list.separator ', ': Prints', 'between each element.join_with soft_newline: Instead of a fixed separator, allows elements to be joined with a flexible line break (e.g., for statements in a block).indented: Each item in the list will be indented relative to thelistdirective's starting point.wrap_after_count: Hint for intelligent line wrapping.
4. Optional Elements (
[ ]):To print an element only if it exists (i.e., the AST field is not null/empty), use square brackets.
[ 'else' space block { list $ElseStatements { $element } } ]The content within the brackets is only rendered if the AST field corresponding to the first referenced element (or a condition if specified) is present.
5. Conditional Formatting (
if):For more complex conditional rendering based on properties or types of an AST node, use
if. Conditions can inspect field values or types.if $IsInline { // $IsInline could be a boolean field or a SATL-computed property '(' $Expression ')' } else { block { $Expression } }- Conditions can also call SATL helper functions:
if call SATL.ShouldWrap($Node) { ... }.
6. SATL Hooks (
call):For very complex formatting logic that cannot be expressed declaratively in PDL (e.g., dynamic indentation based on content, complex heuristic line breaking), SATL functions can be invoked directly. The SATL function should return a string to be inserted into the output.
print MyComplexNode { 'Start: ' call SATL.FormatComplexPart($FirstField, $SecondField) newline call SATL.GenerateIndentedCode($CodeBlock) }The
calldirective ensures seamless integration, allowing PDL to delegate to the full power of SATL when necessary.7. Comments:
Single-line comments starting with
//are supported.Example (Fragment of GrammarForge's own PDL for its GDL grammar):
// --- PDL for a GrammarFile --- print GrammarFile { list $Directives join_with newline { $element } newline newline // Two newlines between directives and rules list $Rules join_with newline { $element } } // --- PDL for a RuleDecl --- print RuleDecl { $Name soft_space '::=' soft_space $Body ';' newline } // --- PDL for RuleBody --- print RuleBody { list $Alternatives separator soft_space '|' { $element } } // --- PDL for RuleAlternation (just prints its sequences) --- print RuleAlternation { list $Sequences join_with soft_space { $element } } // --- PDL for RuleSequence --- print RuleSequence { list $Terms join_with soft_space { $element } } // --- PDL for RuleElement --- print RuleElement { [ $Name ':' soft_space ] // Optional named field $Content [ $Modifier ] // Optional modifier ('?', '*', '+') } // --- PDL for Modifier (just prints the kind or complex count) --- print RuleModifier { if $Kind == "count" { '{' $Count '}' } else { $Kind } }Relationship with other components:
- ADL: PDL directly consumes ADL-defined AST node types and their fields. The entire pretty-printing process is driven by traversing an ADL-structured AST.
- SATL: Provides the imperative escape hatch for complex formatting logic via
calldirectives and for implementing helper functions (PrettyPrint()methods on AST nodes) that PDL can utilize. SATL can also orchestrate the entire pretty-printing process, initiating it from the root AST node. - Self-Hosting: GrammarForge will define its own PDL rules (as shown in the example above) to pretty-print its GDL, ADL, and SATL structures, demonstrating its self-describing nature and proving the utility of PDL.
This declarative and extensible approach makes PDL a powerful tool for maintaining code quality and consistency across various languages defined and managed by GrammarForge.
Prompt for Review: I've detailed the Pretty-Printer Description Language (PDL). Please review this section before I proceed with the final summary and the single-file source code structure.
- User
Please proceed.
Assistant
Alright, we've covered the core DSLs. Let's wrap up with a summary of the holistic design, the OOP flavor, and the critical concept of the single-file source code and self-hosting.
Section 5: Holistic Design, OOP Flavor, and Single-File Source Code
This section brings together all the previously described components, showing how they integrate into a cohesive system, emphasizing the OOP nature, and detailing the unique single-file self-hosting architecture of GrammarForge.
1. Holistic Design: The Interplay of Components
GrammarForge is designed as an integrated ecosystem where each DSL and the scripting language plays a crucial, interconnected role in defining a programming language:
- GDL (Grammar Definition Language): Defines the concrete syntax (how the language looks textually). It provides the raw input for the parser. Its rules implicitly inform the initial structure of the AST.
- ADL (Abstract Syntax Tree Definition Language): Defines the abstract syntax (the conceptual structure of programs). It provides a strongly typed schema for the AST nodes, often derived from GDL but allowing for refinement and explicit polymorphic relationships (interfaces). This is the canonical representation of a program within GrammarForge.
- SATL (Semantic Action & Transformation Language): This Go-like scripting language is the dynamic "brains" of GrammarForge.
- It implements the semantic actions that run during parsing, constructing ADL-defined AST nodes from the GDL-parsed tokens.
- It defines methods on ADL AST nodes for operations like type checking, analysis, transformation, and most importantly, evaluation/interpretation.
- It provides meta-programming capabilities to introspect the GDL/ADL schemas, enabling powerful tooling.
- It provides hooks for advanced pretty-printing.
- PDL (Pretty-Printer Description Language): Defines how ADL-structured ASTs are serialized back to text. It dictates layout, indentation, and spacing, ensuring consistent output. It can leverage SATL for complex formatting.
The Workflow Loop:
- Define GDL: Write the grammar.
- Define ADL: Structure the AST (often a refinement of GDL's implied structure).
- Implement SATL (Parsing): Write semantic actions to build ADL ASTs from GDL rules.
- Implement SATL (Semantics): Write AST visitors, transformers, and the evaluator using ADL types.
- Define PDL: Write pretty-printing rules to go from ADL ASTs back to text.
This cyclical process is key to GrammarForge's power: defining a language means defining all these aspects, often iteratively, within the same coherent environment.
2. OOP Flavor: Building Extensible Language Structures
The OOP flavor in GrammarForge is primarily manifested through the structure and manipulation of AST nodes, providing a robust and extensible foundation for language design:
- AST Nodes as Objects: Every construct defined in ADL (e.g.,
FunctionDecl,BinaryOp,Literal) is treated as an object. These objects have fields (properties) as defined in ADL and can have methods attached to them. - ADL Interfaces (Polymorphism): ADL's
interfacedeclaration is the cornerstone of polymorphism.- An
Expressioninterface can be implemented byLiteral,BinaryOp,CallExpr, etc. - This allows for generic algorithms (e.g., an AST visitor or an
Evaluatemethod) to operate on theExpressioninterface without knowing the concrete type at compile time, dispatching to the correct implementation at runtime. - This maps directly to standard OOP patterns for handling diverse but related data structures.
- An
- SATL Methods: The
func (receiver *Type) MethodName(...)syntax in SATL allows developers to define behavior directly on ADL-defined AST nodes. This is crucial for:- Implementing the
Evaluate()method for an interpreter (e.g.,BinaryOp.Evaluate(),IfStatement.Evaluate()). - Implementing AST traversal logic (e.g.,
Node.VisitChildren()). - Providing context-sensitive pretty-printing (e.g.,
Node.ToDebugString()).
- Implementing the
- Encapsulation (Implicit): While explicit access modifiers aren't a primary focus for AST nodes (which are data structures), the clear separation of concerns between ADL (structure) and SATL (behavior) promotes a form of encapsulation. The internal workings of an
Evaluatemethod for aBinaryOpare self-contained within that method. - Composition over Inheritance (Go-like): While ADL supports
interfacefor polymorphism, the primary way to build complex AST nodes is through composition (e.g., aFunctionCallnode contains anExpressionfor its callee and a list ofExpressions for its arguments), mirroring Go's preference for composition.
This OOP approach makes ASTs not just passive data structures but active participants in the language's semantics, enabling clean, modular, and extensible language implementations.
3. Single-File Source Code and Self-Hosting
This is a defining architectural principle of GrammarForge, designed to showcase its power and simplify distribution.
The
.gfFile Structure:A GrammarForge source file (
.gf) is a single, unified text file that contains the complete definition of a language (or of GrammarForge itself). It is structured using special directives or comment blocks to delineate the different DSL sections:// --- @gdl --- // Grammar Definition Language for MyLanguage // Define tokens and production rules here. Identifier ::= `[a-zA-Z_][a-zA-Z0-9_]*` ; FunctionDecl ::= 'func' name:Identifier params:ParameterList body:Block ; // ... more GDL rules ... // --- @adl --- // Abstract Syntax Tree Definition Language for MyLanguage // Define AST node structures and interfaces here. type Program struct { Declarations []Declaration } interface Expression {} // Implemented by Literal, BinaryOp, CallExpr type FunctionDecl struct { Name string Params []Parameter Body Block } implements Declaration; // ... more ADL types ... // --- @pdl --- // Pretty-Printer Description Language for MyLanguage // Define how AST nodes are printed back to source. print FunctionDecl { 'func' soft_space $Name $Params block { $Body } } print BinaryOp { $Left soft_space $Operator soft_space $Right } // ... more PDL rules ... // --- @satl --- // Semantic Action & Transformation Language (Go-like scripting) for MyLanguage // Implement parser actions, AST methods, evaluation logic, and utility functions. // Parser action for FunctionDecl GDL rule func CreateFunctionDecl(name ast.Identifier, params ast.ParameterList, body ast.Block) *ast.FunctionDecl { return &ast.FunctionDecl{ Name: name.Value, // Assuming Identifier has a Value field Params: params.Params, Body: body, } } // Evaluate method for BinaryOp AST node func (op *ast.BinaryOp) Evaluate(ctx *EvalContext) (Value, error) { // ... evaluation logic ... } // Utility function func MyUtilityFunc(arg string) string { return "Processed: " + arg } // ... more SATL code ...Bootstrapping and The Minimal Runtime:
- Minimal Runtime: GrammarForge requires a small, minimal "bootstrap" runtime (potentially written in a low-level language like Go, Rust, or C). This runtime is not part of the
.gffile itself. - Initial Parse: This runtime's primary job is to load the single
grammarforge.gffile (which defines GrammarForge's own grammar, AST, pretty-printer, and initial SATL logic). - Self-Definition: The runtime first uses a very basic, hardcoded "meta-parser" to recognize the
@gdl,@adl,@pdl, and@satlsections. It then feeds the content of these sections to the appropriate internal processors:- The GDL content is used to generate GrammarForge's own internal parser.
- The ADL content is used to define GrammarForge's own internal AST types (for representing GDL, ADL, PDL, and SATL structures).
- The SATL content (specifically the semantic actions for GrammarForge's own grammar rules) becomes the logic for building GrammarForge's internal representation of its own definition.
- The PDL content is used to generate GrammarForge's own pretty-printer.
- Full System Online: Once this initial bootstrap is complete, the full GrammarForge system is "alive." It can then load other
.gffiles (defining new languages) using its newly generated, self-defined parser, generate parsers/pretty-printers/interpreters for those languages, and execute SATL scripts.
Why Single File and Self-Hosting?
- Portability: A single
.gffile makes it incredibly easy to share and distribute a complete language definition. - Self-Containment: All aspects of a language are in one place, reducing configuration complexity and ensuring consistency.
- Ultimate Demonstration: By defining itself in its own format, GrammarForge provides the most compelling proof of concept for its capabilities. It demonstrates its flexibility, expressiveness, and the power of meta-programming.
- Understandability: A complete, runnable example of how to use GrammarForge is always available by simply looking at
grammarforge.gf. - Bootstrapping Elegance: The circular dependency (language defines language) is elegantly managed by the minimal bootstrap runtime.
This integrated and self-describing architecture makes GrammarForge a uniquely powerful tool for language design, where the language itself is a living example of its own principles.
Prompt for Review: I've now provided the comprehensive plan and detailed each of the sections that comprise GrammarForge. Please review the final section and the overall design.
- User
Please proceed.
Assistant
Excellent! We have now completed the comprehensive design of GrammarForge, a programming language specifically crafted for the purpose of designing other programming languages.
To recap, GrammarForge is built on the principles of self-description, generativity, and introspection, enabled by its core components:
- Grammar Definition Language (GDL): For formally defining concrete syntax with an EBNF-like notation, tightly integrated with AST node creation.
- Abstract Syntax Tree Definition Language (ADL): For declaratively defining the precise structure and polymorphic relationships (OOP flavor) of AST nodes.
- Semantic Action & Transformation Language (SATL): The imperative, Go-like scripting language for implementing parsing actions, AST transformations, meta-programming, and the core evaluation semantics of any defined language. It's the operational heart.
- Pretty-Printer Description Language (PDL): For declaratively specifying how ASTs are transformed back into well-formatted source code, with hooks for SATL for advanced formatting.
The entire system is designed around a single-file source code paradigm, where all GDL, ADL, PDL, and SATL definitions for a given language (including GrammarForge itself) reside in one
.gffile. This is crucial for its self-hosting capabilities, allowing GrammarForge to define, parse, transform, and pretty-print its own source code, serving as the ultimate demonstration of its power and flexibility.This design provides a robust, extensible, and intuitive framework for anyone looking to build, iterate on, or deeply understand programming languages.
Thank you for guiding me through this design process!
- User
Write the complete and fully working single file source code.
Assistant
You've hit upon the ultimate challenge in language design – the self-hosting, self-describing system!
It's crucial to clarify that providing a "complete and fully working" single-file source code for a self-hosting language like GrammarForge is an immense task, equivalent to writing a full compiler and interpreter project from scratch (easily thousands of lines of code). This is far beyond what can be realistically generated in a single response.
What I can provide is a highly detailed and representative skeleton of
grammarforge.gf. This skeleton will contain:- The meta-structure: How the
@gdl,@adl,@pdl, and@satlsections are delimited. - Core GDL for GrammarForge: The grammar rules for defining other GDL rules, ADL types, PDL rules, and basic SATL constructs. This is GrammarForge describing its own syntax.
- Core ADL for GrammarForge: The AST node definitions for GrammarForge's own internal representation of grammars, types, pretty-printing rules, and SATL code.
- Core PDL for GrammarForge: How GrammarForge would pretty-print its own GDL, ADL, etc.
- Representative SATL snippets: Examples of semantic actions for parsing GrammarForge's own definitions, and hints at the kind of meta-programming and introspection that would be possible.
Important Considerations for "Fully Working":
- The Bootstrap Runtime: This
.gffile is not directly executable. It requires a small, external "bootstrap" runtime (e.g., written in Go, Rust, C++) whose sole purpose is to:- Parse this
grammarforge.gffile (recognizing the@gdl,@adl, etc. markers). - Process GrammarForge's own GDL section to build GrammarForge's internal parser.
- Process GrammarForge's own ADL section to define GrammarForge's internal AST types.
- Initialize GrammarForge's own SATL environment and load the SATL code within this file.
- Bring the full GrammarForge system online, allowing it to then load other
.gffiles or interpret SATL scripts.
- Parse this
- Full Implementation Complexity: Implementing a full parser generator, an ADL type system compiler, a PDL layout engine, and especially a Go-like interpreter for SATL (which can then interpret more SATL code) are all massive tasks that cannot be condensed into snippets. My SATL examples will focus on what they would achieve, rather than the full, intricate logic.
Here is the structured skeleton of
grammarforge.gf, demonstrating how GrammarForge would define itself:// grammarforge.gf - The Self-Defining Source Code for GrammarForge // This file defines GrammarForge itself, using its own DSLs. // It is processed by a minimal external "bootstrap" runtime (e.g., in Go/Rust/C#). // The bootstrap runtime reads these sections, builds GrammarForge's core components, // and brings the full system online. // --- @gdl --- // GDL (Grammar Definition Language) Section for GrammarForge itself. // This defines the syntax of GrammarForge's own language (how GDL, ADL, PDL, SATL are written). %start GrammarForgeFile; // Global structure of a GrammarForge file GrammarForgeFile ::= { Section } ; Section ::= CommentBlock:comment | GDLSection:gdl | ADLSection:adl | PDLSection:pdl | SATLSection:satl ; CommentBlock ::= "// ---" Whitespace* Identifier:tag Whitespace* "---" (Whitespace | Newline) { Line } ; Line ::= { ~Newline Char } Newline ; // Any characters until a newline Char ::= `.` ; // Any character GDLSection ::= "// --- @gdl ---" Newline { GDLDirective | GDLRuleDecl | GDLTokenDecl } ; ADLSection ::= "// --- @adl ---" Newline { ADLTypeDecl } ; PDLSection ::= "// --- @pdl ---" Newline { PDLPrintRule } ; SATLSection ::= "// --- @satl ---" Newline { SATLFuncDecl | SATLVarDecl | SATLStatement } ; // Simplified // --- GDL Language Definition (GrammarForge's GDL syntax) --- // This part defines the syntax of the GDL language itself. GDLDirective ::= '%' ( GDLStartDirective | GDLImportDirective ) ';' ; GDLStartDirective ::= 'start' ruleName:Identifier ; GDLImportDirective ::= 'import' path:StringLiteral ; GDLRuleDecl ::= ruleName:Identifier '::=' ruleBody:GDLRuleBody ';' ; GDLTokenDecl ::= tokenName:Identifier '::=' regex:RegexLiteral [ '->' 'skip' ] ';' ; GDLRuleBody ::= ( term:GDLRuleTerm { op:('|') term:GDLRuleTerm } ) // Alternatives | GDLRuleTerm // Single term ; GDLRuleTerm ::= element:GDLElement [ modifier:GDLModifier ] ; GDLElement ::= name:Identifier ':' element:GDLElement // Named field | literal:StringLiteral | regex:RegexLiteral | ruleRef:Identifier // Reference to another rule | '(' groupBody:GDLRuleBody ')' // Grouping ; GDLModifier ::= '?' | '*' | '+' | '{' count:IntegerLiteral '}' // Exact count ; // --- ADL Language Definition (GrammarForge's ADL syntax) --- // This part defines the syntax of the ADL language itself. ADLTypeDecl ::= type:('type' | 'interface') typeName:Identifier [ 'inherits' superClass:Identifier ] // Simplified single inheritance/interface '{' { ADLFieldDecl } '}' [ 'implements' { interfaceName:Identifier } ] // Implements multiple interfaces ';' ; ADLFieldDecl ::= fieldName:Identifier fieldType:ADLFieldType ';' ; ADLFieldType ::= '[' ']' elementType:Identifier // List type | baseType:Identifier // Primitive or ADL type reference ; // --- PDL Language Definition (GrammarForge's PDL syntax) --- // This part defines the syntax of the PDL language itself. PDLPrintRule ::= 'print' astNodeName:Identifier '{' { PDLElement } '}' ; PDLElement ::= literal:StringLiteral | '$' fieldName:Identifier // Field reference | 'space' | 'soft_space' | 'no_space' | 'newline' | 'soft_newline' | 'indent' | 'dedent' | 'block' | listDirective:PDLListDirective | optionalBlock:PDLOptionalBlock | conditionalBlock:PDLConditionalBlock | callDirective:PDLCallDirective ; PDLListDirective ::= 'list' '$' fieldName:Identifier { PDLListOption } '{' { PDLElement } '}' ; PDLListOption ::= 'separator' StringLiteral | 'each' '{' { PDLElement } '}' | 'join_with' ( 'newline' | 'soft_newline' | 'space' | 'soft_space' ) | 'indented' | 'wrap_after_count' IntegerLiteral ; PDLOptionalBlock ::= '[' { PDLElement } ']' ; PDLConditionalBlock ::= 'if' condition:SATLExpression '{' { PDLElement } '}' [ 'else' '{' { PDLElement } '}' ] ; PDLCallDirective ::= 'call' SATLCallExpr ; // Call a SATL function // --- SATL Language Definition (GrammarForge's SATL syntax - highly simplified) --- // This part defines the syntax of a small subset of the SATL language itself. // A full Go-like grammar is extremely complex. SATLFuncDecl ::= 'func' [ '(' receiverName:Identifier '*' receiverType:Identifier ')' ] funcName:Identifier '(' { SATLParam } ')' [ '(' { SATLParam } ')' ] '{' { SATLStatement } '}' ; SATLParam ::= paramName:Identifier paramType:Identifier ; SATLVarDecl ::= 'var' varName:Identifier varType:Identifier [ '=' SATLExpression ] ';' ; SATLStatement ::= SATLVarDecl | SATLAssignStmt ';' | SATLCallExpr ';' | 'return' [ SATLExpression ] ';' | 'if' SATLExpression '{' { SATLStatement } '}' [ 'else' '{' { SATLStatement } '}' ] | 'for' SATLExpression '{' { SATLStatement } '}' // simplified loop | '{' { SATLStatement } '}' // Block ; SATLAssignStmt ::= Identifier '=' SATLExpression ; SATLExpression ::= SATLCallExpr | StringLiteral | IntegerLiteral | Identifier ; // Very basic expression, e.g. function call, literal or variable SATLCallExpr ::= Identifier '(' { SATLExpression [ ',' SATLExpression ] } ')' ; // func(arg1, arg2) // --- Common Lexical Tokens (Used across GDL, ADL, PDL, SATL) --- Identifier ::= `[a-zA-Z_][a-zA-Z0-9_]*` ; StringLiteral ::= `\"([^"\\]|\\.)*\"` ; // e.g., "hello", "func" RegexLiteral ::= '`' (`[^`\\]|\\.`)* '`' ; // e.g., `[a-zA-Z_]*` IntegerLiteral ::= `[0-9]+` ; Whitespace ::= `\s+` -> skip ; Newline ::= `\n` -> skip ; // Newline tokens are skipped as they primarily define structure Comment ::= `//.*` -> skip | `/\*(\*(?!/)|[^*])*\*/` -> skip ; // Single-line and multi-line comments are skipped // --- @adl --- // ADL (Abstract Syntax Tree Definition Language) Section for GrammarForge itself. // This defines the structure of the AST nodes that represent GrammarForge's own definitions. // Base interface for all GrammarForge AST nodes interface GrammarForgeNode {} // Top-level structure of a .gf file type GrammarForgeFile struct { Sections []Section } implements GrammarForgeNode; interface Section {} type GDLSection struct { Directives []GDLDirective Rules []GDLRuleDecl Tokens []GDLTokenDecl } implements Section; type ADLSection struct { TypeDecls []ADLTypeDecl } implements Section; type PDLSection struct { PrintRules []PDLPrintRule } implements Section; type SATLSection struct { FuncDecls []SATLFuncDecl VarDecls []SATLVarDecl // Global vars Statements []SATLStatement // Top-level statements } implements Section; type CommentBlock struct { Tag string // e.g., "gdl", "adl", "satl" Lines []string } implements Section; // --- GDL ADL Nodes --- interface GDLDirective {} type GDLStartDirective struct { RuleName string } implements GDLDirective; type GDLImportDirective struct { Path string } implements GDLDirective; type GDLRuleDecl struct { Name string Body GDLRuleBody } implements GrammarForgeNode; type GDLTokenDecl struct { Name string Regex string Skip bool // true if '-> skip' is present } implements GrammarForgeNode; interface GDLRuleBody {} type GDLAlternationBody struct { Terms []GDLRuleTerm // Each term is an alternative } implements GDLRuleBody; type GDLSequenceBody struct { Terms []GDLRuleTerm // Terms in sequence } implements GDLRuleBody; type GDLRuleTerm struct { Element GDLTermElement Modifier *GDLModifier // Optional modifier } implements GrammarForgeNode; interface GDLTermElement {} type GDLNamedElement struct { Name string Element GDLTermElement } implements GDLTermElement; type GDLStringLiteral struct { Value string } implements GDLTermElement; type GDLRegexLiteral struct { Value string } implements GDLTermElement; type GDLRuleRef struct { Name string } implements GDLTermElement; type GDLGroupedBody struct { Body GDLRuleBody } implements GDLTermElement; type GDLModifier struct { Kind string // e.g., "?", "*", "+", "count" Count int // If Kind is "count" } implements GrammarForgeNode; // --- ADL ADL Nodes --- interface ADLTypeDecl {} // Base interface for type and interface declarations type ADLStructDecl struct { Name string Fields []ADLFieldDecl Implements []string // List of interface names } implements ADLTypeDecl; type ADLInterfaceDecl struct { Name string Inherits []string // List of interface names this one inherits from (if multiple inheritance/composition is added later) // No fields in interfaces themselves } implements ADLTypeDecl; type ADLFieldDecl struct { Name string Type ADLFieldType } implements GrammarForgeNode; type ADLFieldType struct { IsList bool BaseType string // Reference to another ADL type name (e.g., "Expression", "string") } implements GrammarForgeNode; // --- PDL ADL Nodes --- type PDLPrintRule struct { AstNodeName string Elements []PDLElement } implements GrammarForgeNode; interface PDLElement {} type PDLLiteral struct { Value string // The literal string (e.g., "func", " ") } implements PDLElement; type PDLFieldRef struct { FieldName string } implements PDLElement; type PDLKeyword struct { // For "space", "newline", "indent" etc. Kind string } implements PDLElement; type PDLListDirective struct { FieldName string Separator *string // optional string literal for separator JoinWith *string // optional keyword for join type (newline, soft_space etc) Indented bool WrapAfter *int // optional integer count for wrapping Elements []PDLElement // Elements to print for each item } implements PDLElement; type PDLOptionalBlock struct { Elements []PDLElement } implements PDLElement; type PDLConditionalBlock struct { Condition SATLExpression // Simplified to an SATL expression (e.g., a call) IfBranch []PDLElement ElseBranch []PDLElement // Optional } implements PDLElement; type PDLCallDirective struct { Call SATLCallExpr // Call a SATL function } implements PDLElement; // --- SATL ADL Nodes (Highly Simplified) --- type SATLFuncDecl struct { ReceiverName string // If it's a method ReceiverType string Name string Params []SATLParam ReturnParams []SATLParam // Return values (Go-style tuple) Body []SATLStatement } implements GrammarForgeNode; type SATLParam struct { Name string Type string } implements GrammarForgeNode; interface SATLStatement {} type SATLVarDecl struct { Name string Type string InitExpr *SATLExpression // Optional initialization } implements SATLStatement; type SATLAssignStmt struct { Target string ValueExpr SATLExpression } implements SATLStatement; type SATLReturnStmt struct { Value *SATLExpression // Optional return value } implements SATLStatement; type SATLIfStmt struct { Condition SATLExpression IfBody []SATLStatement ElseBody []SATLStatement // Optional } implements SATLStatement; type SATLForStmt struct { Condition SATLExpression // Loop condition Body []SATLStatement } implements SATLStatement; type SATLBlockStmt struct { Statements []SATLStatement } implements SATLStatement; interface SATLExpression {} type SATLCallExpr struct { FunctionName string Args []SATLExpression } implements SATLExpression; type SATLStringLiteral struct { Value string } implements SATLExpression; type SATLIntegerLiteral struct { Value int } implements SATLExpression; type SATLIdentifier struct { Value string } implements SATLExpression; // --- @pdl --- // PDL (Pretty-Printer Description Language) Section for GrammarForge itself. // This defines how to pretty-print GrammarForge's own GDL, ADL, PDL, and SATL. print GrammarForgeFile { list $Sections join_with newline { $element } } print GDLSection { '// --- @gdl ---' newline list $Directives join_with newline { $element } newline // Space between directives and rules list $Rules join_with newline { $element } newline // Space between rules and tokens list $Tokens join_with newline { $element } } print ADLSection { '// --- @adl ---' newline list $TypeDecls join_with newline { $element } } print PDLSection { '// --- @pdl ---' newline list $PrintRules join_with newline { $element } } print SATLSection { '// --- @satl ---' newline list $FuncDecls join_with newline { $element } newline // Space between funcs and vars list $VarDecls join_with newline { $element } newline // Space between vars and statements list $Statements join_with newline { $element } } print GDLRuleDecl { $Name soft_space '::=' soft_space $Body ';' } print GDLTokenDecl { $Name soft_space '::=' soft_space $Regex [ '->' soft_space 'skip' ] ';' } print GDLAlternationBody { list $Terms separator soft_space '|' { $element } } print GDLSequenceBody { list $Terms join_with soft_space { $element } } print GDLRuleTerm { $Element [ $Modifier ] } print GDLNamedElement { $Name ':' soft_space $Element } print GDLStringLiteral { $Value } print GDLRegexLiteral { '`' $Value '`' } // Use explicit backticks for regex print GDLRuleRef { $Name } print GDLGroupedBody { '(' $Body ')' } print GDLModifier { if $Kind == "count" { '{' $Count '}' } else { $Kind } } print ADLStructDecl { 'type' soft_space $Name soft_space 'struct' soft_space '{' indent newline list $Fields join_with newline { $element } dedent newline '}' [ soft_space 'implements' soft_space list $Implements separator soft_space ',' { $element } ] ';' } print ADLInterfaceDecl { 'interface' soft_space $Name soft_space '{' indent newline // No fields in interface directly dedent newline '}' [ soft_space 'inherits' soft_space list $Inherits separator soft_space ',' { $element } ] ';' } print ADLFieldDecl { $Name soft_space $Type } print ADLFieldType { [ '[' ']' ] $BaseType } print PDLPrintRule { 'print' soft_space $AstNodeName soft_space '{' indent newline list $Elements join_with newline { $element } dedent newline '}' } print PDLListDirective { 'list' soft_space '$' $FieldName [ 'separator' soft_space $Separator ] [ 'join_with' soft_space $JoinWith ] [ 'indented' ] [ 'wrap_after_count' soft_space $WrapAfter ] block { list $Elements { $element } } } print PDLOptionalBlock { '[' list $Elements { $element } ']' } print PDLConditionalBlock { 'if' soft_space $Condition soft_space block { list $IfBranch { $element } } [ 'else' soft_space block { list $ElseBranch { $element } } ] } print PDLCallDirective { 'call' soft_space $Call } print SATLFuncDecl { 'func' [ '(' $ReceiverName soft_space '*' $ReceiverType ')' soft_space ] $Name '(' list $Params separator ',' { $element.Name soft_space $element.Type } ')' [ '(' list $ReturnParams separator ',' { $element.Name soft_space $element.Type } ')' ] block { list $Body { $element } } } print SATLParam { $Name soft_space $Type } print SATLVarDecl { 'var' soft_space $Name soft_space $Type [ soft_space '=' soft_space $InitExpr ] ';' } print SATLAssignStmt { $Target soft_space '=' soft_space $ValueExpr ';' } print SATLReturnStmt { 'return' [ soft_space $Value ] ';' } print SATLIfStmt { 'if' soft_space $Condition soft_space block { list $IfBody { $element } } [ 'else' soft_space block { list $ElseBody { $element } } ] } print SATLForStmt { 'for' soft_space $Condition soft_space block { list $Body { $element } } } print SATLBlockStmt { block { list $Statements { $element } } } // For generic blocks print SATLCallExpr { $FunctionName '(' list $Args separator ',' { $element } ')' } print SATLStringLiteral { $Value } print SATLIntegerLiteral { $Value } print SATLIdentifier { $Value } // --- @satl --- // SATL (Semantic Action & Transformation Language) Section for GrammarForge itself. // This code is run by the bootstrap runtime to build GrammarForge's internal representations // and define its core operations. // This section would contain the *actual code* that implements: // 1. Semantic actions to build the ADL ASTs for the GDL/ADL/PDL/SATL definitions above. // 2. The parser generator logic (reads GDL AST, outputs a parser). // 3. The ADL type system compiler (reads ADL AST, defines internal types for SATL). // 4. The PDL pretty-printer engine (reads PDL AST, applies formatting). // 5. The SATL interpreter/compiler (enables execution of user SATL code). // 6. The introspection API (`GF.Schema`). // Example: Semantic action for parsing a GDL Rule Declaration // (This is what the bootstrap runtime's initial parser would call when it sees a GDLRuleDecl) func CreateGDLRuleDecl(name ast.Identifier, body ast.GDLRuleBody) *ast.GDLRuleDecl { // In a real system, 'ast.Identifier' would be an AST node representing the identifier // For this example, we assume it has a '.Value' field. return &ast.GDLRuleDecl{ Name: name.Value, Body: body, } } // Example: Semantic action for parsing a GDL Token Declaration func CreateGDLTokenDecl(tokenName ast.Identifier, regex ast.RegexLiteral, skipToken *ast.Token) *ast.GDLTokenDecl { // 'skipToken' would be non-nil if '-> skip' was present in the GDL. return &ast.GDLTokenDecl{ Name: tokenName.Value, Regex: regex.Value, Skip: skipToken != nil, } } // Example: Semantic action for an ADL Struct Declaration func CreateADLStructDecl(typeName ast.Identifier, fields []ast.ADLFieldDecl, implements []ast.Identifier) *ast.ADLStructDecl { implementedInterfaces := []string{} for _, impl := range implements { implementedInterfaces = append(implementedInterfaces, impl.Value) } return &ast.ADLStructDecl{ Name: typeName.Value, Fields: fields, Implements: implementedInterfaces, } } // Example: A method on an AST node type for evaluation (placeholder logic) // This method would be defined on the ADL-defined 'BinaryOp' type. func (op *ast.BinaryOp) Evaluate(ctx *ast.EvalContext) (ast.Value, error) { leftVal, err := op.Left.Evaluate(ctx) if err != nil { return nil, err } rightVal, err := op.Right.Evaluate(ctx) if err != nil { return nil, err } // Placeholder: In reality, complex type checking and operation dispatch would happen here. if op.Operator == "+" { // Assuming integer values return ast.NewIntValue(leftVal.(ast.IntValue).Val + rightVal.(ast.IntValue).Val), nil } // ... more operators ... return nil, fmt.Errorf("unsupported operator: %s", op.Operator) } // Example: A meta-programming function for introspection // This function accesses the global GrammarForge schema. func GetRuleNamesWithModifier(modifier string) []string { matchingRules := []string{} // GF.Schema.GDL would be the AST representation of GrammarForge's own GDL definitions. for _, rule := range GF.Schema.GDL.Rules { // This logic is simplified; it would traverse rule.Body to find elements with modifiers. // Assume rule.Body.Terms directly contains elements with modifiers for this example. for _, term := range rule.Body.Terms { // Assuming a flat structure for simplicity if term.Modifier != nil && term.Modifier.Kind == modifier { matchingRules = append(matchingRules, rule.Name) break // Found for this rule, move to next } } } return matchingRules } // Placeholder for main entry point (would be called by the bootstrap runtime) func Main() { println("GrammarForge Bootstrap Complete!") println("Successfully parsed and initialized GrammarForge's own definitions.") // Here, the system would typically expose a REPL, // or take command-line arguments to load/parse/evaluate user .gf files. // Example of using introspection: rulesWithOptional := GetRuleNamesWithModifier("?") println(fmt.Sprintf("Rules with optional elements: %v", rulesWithOptional)) // Example of defining a simple language (hypothetical, simplified) // In a real scenario, this would be loaded from a separate .gf file. // parseResult := GF.ParseString("my_language_code.mylang", "func main() { print 'Hello' }") // if parseResult.Err != nil { // println(fmt.Sprintf("Parse error: %v", parseResult.Err)) // } else { // println("AST parsed successfully. Evaluating...") // // GF.Evaluate(parseResult.AST) // Assuming a top-level evaluate function // println("Done.") // } } // Bootstrap calls Main() // (This is a conceptual call; the actual bootstrap runtime would invoke Main or similar) // Main()- The meta-structure: How the
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.