Skip to content

Commit 82e7e04

Browse files
Fix DateTime test failures: overload warnings and custom warning categories (#352)
* Fix Module::Runtime test failures: #line directive, hints hash, reload message Three fixes that reduce Module::Runtime test failures from 23 to 8: 1. Honor #line directive in use statement caller info - parseUseDeclaration now uses getSourceLocationAccurate() to get the #line-adjusted filename and line number for CallerStack.push() - Fixes t/import_error.t tests where eval'd use statements with #line directives were reporting wrong locations 2. Prevent %^H hints hash from leaking into require'd modules - doFile() now saves, clears, and restores %^H around PerlLanguageProvider.executePerlCode() - In Perl >= 5.11 (which we emulate), hints don't leak into required files - Fixes tests that check $^H{...} is undef in BEGIN blocks of required modules 3. Fix cached require failure error message - Changed 'Compilation failed in require at <file>' to 'Attempt to reload <file> aborted.' - Matches Perl's actual error message for cached compilation failures - Fixes the 'broken module is visibly broken when re-required' tests Remaining 8 failures are due to caller()[10] (hints hash per stack frame) returning undef - this is a known limitation requiring more complex tracking. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix base.pm isa check and error message formatting - Base.java: Add isa check before adding to @isa, matching Perl base.pm behavior (skip redundant base classes when Middle->isa(Parent)) - PerlCompilerException.java, FileTestOperator.java: Add missing period before " at file line N" in error messages Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix parent.pm tests: normalize old-style package separator and improve error messages - NameNormalizer: Add normalizePackageName() to convert Foo'Bar to Foo::Bar - InheritanceResolver, DFS: Normalize package names when reading @isa - Universal.isa: Normalize argument for consistent comparison - ModuleOperators: Include module name hint and @inc entries in "Can't locate" error message, matching Perl 5.17.5+ behavior All 8 parent.pm tests now pass. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix Module::Metadata tests: Unicode regex, File::Spec path handling - RegexFlags: Enable UNICODE_CHARACTER_CLASS so \w, \d, \s match Unicode characters by default (matches Perl behavior) - FileSpec.abs2rel: Fix to use user.dir property for relative base paths (Java Path.toAbsolutePath() ignores System.setProperty changes) - FileSpec.rel2abs: Same fix for relative base paths Module::Metadata tests: 137/138 pass (1 taint test expected to fail) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix %main:: to include top-level packages in stash enumeration In Perl, $Foo::x and $main::Foo::x refer to the same variable, but PerlOnJava stores top-level package symbols without the 'main::' prefix. This caused %main:: (the main stash) to not include entries like 'Foo::' for top-level packages. The fix extends HashSpecialVariable.entrySet() to also include keys that start with a top-level package name (e.g., "Foo::test") when enumerating %main::. This allows Class::Inspector::_subnames to correctly find all child packages. Test results: - Class::Inspector: 55/56 tests pass (1 failure is unrelated INC hook issue) - All unit tests pass Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix substr() with negative offsets that overshoot string start When substr() is called with a negative offset that goes before the beginning of the string, Perl's behavior is: 1. If the adjusted length would still be positive, clip offset to 0 and reduce length by the overshoot amount (no warning) Example: substr("a", -2, 2) returns "a" 2. If the adjusted length would be non-positive, warn and return undef Example: substr("hello", -10, 1) warns and returns undef This also fixes the 4-argument substr replacement behavior to correctly replace only the extracted portion when clipping occurs. Example: substr("ab", -3, 2, "X") returns "a" and sets str to "Xb" Test results: - All unit tests pass - Class::Inspector tests pass (no more substr outside of string warnings) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix regex /u flag: only enable Unicode character classes when requested Instead of unconditionally enabling UNICODE_CHARACTER_CLASS (which broke 308 tests in re/charset.t), now properly track the /u modifier and only enable Unicode character class matching when /u is specified. This fixes the regressions in: - re/charset.t: 5282/5552 (matches master) - uni/variables.t: 66880/66880 (matches master) - re/regex_sets.t: restored to master level - re/pat.t: restored to master level The /u flag can be used to enable Unicode matching: /\w+/u # matches Unicode word characters Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix cached require error message to include 'Compilation failed' Perl's error message for a cached compilation failure includes both: - 'Attempt to reload <file> aborted.' - 'Compilation failed in require at <file>' The previous fix only included the first part, which broke comp/require.t test 32. Now includes both parts to match Perl. Fixes: comp/require.t 1743/1747 (matches master) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix version qv flag and stringify for decimal versions When a decimal version like '1.0' was passed to version->new(), PerlOnJava was incorrectly setting qv=true and storing 'v1.0' as the original string. This caused CPAN::Meta::Requirements to format versions as '<= v1.0.0' instead of '<= 1.0', breaking CPAN::Meta::Check tests. The fix: - Track the original version string before prepending 'v' for internal use - Set qv=true only if the ORIGINAL input started with 'v' - Store the original input string for stringify(), not the modified one Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix version module: strip trailing zeros and reject math ops 1. Version.java: Strip trailing zeros from double versions - version->new(1.0203) now stringifies to '1.0203' not '1.020300' - version->new(1.23) now stringifies to '1.23' not '1.230000' 2. version.pm: Add overload operators that throw errors for math ops - +, -, *, /, abs, +=, -=, *=, /= now die with 'operation not supported with version object' Version tests: 93.7% -> 99.5% pass rate (220/221 passing) Remaining failures are infrastructure issues: - 02derived.t: File::Temp directory behavior differs - 07locale.t: POSIX::locale_h not implemented Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix File::Temp: TEMPLATE option and PERMS support - Support TEMPLATE => 'nameXXXXXX' as hash option for tempfile/tempdir - Support PERMS => 0400 for custom file permissions - Return open filehandle from _mkstemp_perl to avoid re-open issues - Apply chmod after filehandle is obtained (avoids permission denied) File::Temp tests improved: - tempfile.t: 22/30 pass (cleanup issues due to chdir) - posix.t: 7/7 pass - cmp.t: 18/19 pass - object.t: 28/35 pass Remaining failures are mostly cleanup-related when test uses chdir into temp directory (can't delete directory while in it). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix File::Temp cleanup when chdir'd or using relative paths - Convert paths to absolute when registering for cleanup - Handle cleanup when current directory is the temp dir to be deleted (chdir out before rmtree, like system Perl) - Add _wrap_file_spec_tmpdir() for compatibility - Load Cwd early to avoid CORE::GLOBAL::stat conflicts All 30 tempfile.t tests now pass, including cleanup after chdir. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix anonymous glob slot dereferencing (${*$fh}, %{*$fh}, @{*$fh}) Anonymous globs created by 'open(my $fh, ...)' have a null globName and cannot use GlobalVariable to store their SCALAR, ARRAY, and HASH slots. This commit adds local slot storage for anonymous globs. Changes: - Add scalarSlot, arraySlot, hashSlot private fields to RuntimeGlob - Add getGlobHash() and getGlobArray() methods to RuntimeGlob - Update getGlobSlot() to handle null globName with local slots - Fix scalarDeref(), scalarDerefNonStrict() to use glob.hashDerefGet() - Fix hashDeref(), hashDerefNonStrict() to use glob.getGlobHash() - Fix arrayDeref(), arrayDerefNonStrict() to use glob.getGlobArray() This enables File::Temp OO interface which stores metadata in glob slots. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix File::Temp tests: fileno, autoflush, template path handling Changes: - CustomFileChannel.fileno(): Return synthetic fd instead of undef This allows code checking defined fileno to work correctly - FileTemp.java: Fix argument parsing for _mkstemp/_mkstemps/_mkdtemp Methods now support both function calls and method calls - FileTemp.java: Fix path handling when template prefix ends with / Properly handle templates like /tmp/XXXXXX where the prefix is a directory path with trailing separator - File/Temp.pm: Fix _replace_XX to only replace trailing Xs Previously replaced all Xs in template, now matches Perl 5 behavior - File/Temp.pm: Add autoflush() method for OO interface Uses select/$| to set autoflush on the underlying filehandle - file_temp.t: Fix test for template with only Xs Check basename instead of full path for pattern matching Tests 9 and 12 (Cleanup/destructor) remain failing due to known limitation: PerlOnJava does not call DESTROY when objects go out of scope (Java GC does not support deterministic destruction). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Revert fileno synthetic fd change to fix io/perlio_leaks.t regressions The synthetic fd approach caused regressions in: - io/perlio_leaks.t (12/12 -> 0/12) - io/dup.t (25/29 -> 17/29) - op/require_37033.t (7/10 -> 6/10) These tests rely on fileno returning undef for handles without real fds, since is(undef, undef) passes in comparisons. Updated file_temp.t to check handle validity using ref() instead of fileno() since Java cannot expose real OS file descriptors. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix DateTime test failures: overload warnings and custom warning categories - CompareOperators.java: Add checkSpaceshipResult() to emit "uninitialized value" warning when overloaded <=> returns undef in derived comparison ops - CompareOperators.java: Improve callerWhere() to skip internal Test::* frames for correct warning location reporting - warnings/register.pm: Implement proper warnings::register with import() - WarningFlags.java: Add registerCategory() for runtime custom warning category registration, globalWarningsEnabled flag for runtime scope checks - Warnings.java: Add register_categories(), fix warnif() to use WarnDie.warn() - ScopedSymbolTable.java: Add registerCustomWarningCategory() for bit allocation Fixes t/29overload.t completely. Improves t/46warnings.t (3/6 tests pass). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix indirect object syntax with blocks for undefined barewords Parser changes: - Fix parsing of "bareword { block } args" when bareword is undefined - Correctly parse as indirect object syntax: (block_result)->bareword(args) - This matches Perl behavior for try/catch style constructs without imports Note: t/48rt-115983.t still fails because namespace::autoclean is a stub. The test expects DateTime to clean imported try/catch from its namespace, but implementing autoclean properly causes regressions in other tests. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix warnings::warnif to work with Test::Warnings warning capture - Remove runtime suppression stack that was breaking local $SIG{__WARN__} - Handle no warnings; (without arguments) to disable all warnings - warnings::warnif now properly goes through $SIG{__WARN__} handler - Test::Warnings::warnings { } now captures warnif warnings correctly Note: Lexical warning suppression (no warnings category) works at compile time but does not propagate through module calls at runtime. This is a known limitation requiring future work to pass warning bits through the call stack. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Add warnings scope design doc and infrastructure Phase 1 of lexical warning scope propagation: - Add scope ID tracking to WarningFlags.java - registerScopeWarnings() assigns unique scope IDs - isWarningDisabledInScope() checks if category is suppressed - Design doc in dev/design/warnings-scope.md This enables "no warnings 'DateTime'" to propagate to warnif() calls in DateTime.pm via the local $^WARNING_SCOPE mechanism. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Implement lexical warning scope propagation for warnif() This allows "no warnings 'Category'" in user code to suppress warnings::warnif('Category', $msg) calls in library code (e.g., DateTime.pm). Implementation: - Add ${^WARNING_SCOPE} global variable to track runtime warning scope - noWarnings() registers disabled categories with unique scope IDs - CompilerFlagNode carries scope ID, emits local ${^WARNING_SCOPE} = id - warnIf() checks ${^WARNING_SCOPE} to see if category is suppressed - FindDeclarationVisitor detects scope nodes for proper cleanup Files changed: - WarningFlags.java: Scope ID tracking infrastructure - Warnings.java: Register scopes, check in warnIf() - GlobalContext.java: Initialize ${^WARNING_SCOPE} - CompilerFlagNode.java: Add warningScopeId field - StatementParser.java: Pass scope ID to CompilerFlagNode - EmitCompilerFlag.java: Emit local assignment bytecode - FindDeclarationVisitor.java: Detect scope nodes for cleanup Test: DateTime t/46warnings.t now passes 6/6 (was 3/6) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix runtime warning scope check in RuntimeIO Add isWarningSuppressedAtRuntime() helper to WarningFlags and use it in RuntimeIO to check both compile-time and runtime warning suppression for syscalls warnings (e.g., nul character in pathname). This fixes io/open.t test 192 which tests "no warnings 'syscalls'". Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Add architecture documentation - dev/architecture/README.md: Overview of PerlOnJava architecture - dev/architecture/dynamic-scope.md: local mechanism and DynamicVariableManager - dev/architecture/lexical-pragmas.md: Warnings, strict, and ${^WARNING_SCOPE} These documents explain: - How local saves/restores variable state on scope exit - How the same mechanism is used for defer, regex state, warning scope - How lexical pragmas work at compile-time vs runtime - The ${^WARNING_SCOPE} mechanism for warnif() propagation Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix stringConcatWarnUninitialized to avoid double FETCH on tied scalars The stringConcatWarnUninitialized method was calling getDefinedBoolean() to check for undef values before calling toString(). Both methods trigger FETCH on tied scalars, causing extra FETCH calls. Fix: resolve tied scalars once upfront, then use the resolved value for both the definedness check and the string conversion. This fixes the op/gmagic.t regression (31 tests pass vs 29 before). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix tied variable FETCH/STORE semantics for chained assignments - Add setVoid() method for operations that don't need the assignment result (like chop/chomp) to avoid unnecessary FETCH-after-STORE - For TIED_SCALAR wrapper in RuntimeScalar.set(), delegate to the tied object's set() method instead of doing FETCH-after-STORE directly - Override set() in TieScalar to do FETCH-after-STORE for actual tied scalars (needed for chained assignments like $s = $tied = value) - TiedVariableBase.set() (used by hash/array proxy entries) just does STORE without extra FETCH, which is correct for those cases This fixes the op/gmagic.t concat-assignment tests (37 passing, up from 31) and the tie_hash.t FETCH count test. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix regex /i flag to not affect Unicode properties In Perl, the /i flag makes matching case-insensitive for literals, but does NOT affect Unicode property matching (\p{...}). Changes: - RegexPreprocessorHelper: Wrap \p{...} translations in (?-i:...) to disable case-insensitive matching for property references - RegexPreprocessor: Skip over \p{...}, \P{...}, \N{...}, \x{...}, \o{...} constructs during case-fold expansion to prevent mangling property names (e.g., 'k' in 'Blk' was being expanded) - ExtendedCharClass: Wrap output in (?-i:...) since Perl's (?[...]) applies /i only to literals, not Unicode properties This fixes re/regex_sets.t tests 26-27 and enables many more tests to pass (79/88 up from 25/88). Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Revert tied variable and regex changes that caused test regressions The previous commits introduced regressions in multiple test suites: - op/eval.t: -49 tests (FETCH called when method does not exist) - re/regex_sets_compat.t: -16 tests ((?-i:...) wrapper broke /i flag) - op/bop.t: -9 tests (extra FETCH calls for tied vec operations) - re/subst.t: -7 tests (extra FETCH calls) Reverted changes: - TieScalar: Remove set() override that did STORE+FETCH - RuntimeScalar: Revert FETCH-after-STORE changes for tied scalars - StringOperators: Revert tied scalar concat handling changes - ExtendedCharClass: Remove (?-i:...) wrapper - RegexPreprocessorHelper: Remove (?-i:...) wrapper around \p{...} Test results after fix match or exceed baselines. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Restore StringOperators fix for tied scalar concat (fixes op/gmagic.t) The previous revert accidentally removed the fix that prevents double FETCH on tied scalars during string concatenation with warning checks. This restores the fix from commit 38832fe which resolves tied variables once upfront, then uses the resolved values for both definedness check and string conversion. Test results: - op/gmagic.t: 31/42 (was 29/42 after revert) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix /i flag handling for Unicode properties in extended character classes - /i flag no longer incorrectly affects \p{} property matching - Extended char classes now properly expand literals for case-insensitive matching - Interpolated extended char classes retain their original /i flag setting - /i on outer extended char class does not leak to interpolated inner patterns Fixes: - re/regex_sets.t tests 26, 47-50 now pass (76->81 passing) - Unicode property \p{Blk=ASCII} no longer matches case-folded chars under /i - KELVIN SIGN with /i correctly matches K and k in extended char classes Implementation: - Wrap Unicode property translations in (?-i:...) to protect from /i - Expand all literals (not just special folds) in extended char classes - Track /i flag through nested extended char class processing - Skip extended char classes in expandMultiCharFolds() preprocessing Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix extended char class negation with /i flag Handle ^ at the start of a character class as the negation metacharacter, not as the start of a range. This fixes [^-b] which should mean "not hyphen or b", not "range from ^ to b". Also track atStart flag to properly handle - as a literal when it appears immediately after ^ or at the very start. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix case-insensitive matching for extended char class ranges and escapes - Add LONG S (U+017F) to special case fold mappings - Handle \x{...} escape sequences with case expansion - Handle \N{...} named characters with case expansion for non-special folds - Add special Unicode case folds (KELVIN, LONG S) for ranges like [a-z] This fixes regressions where: - /[a-z]/i did not match KELVIN SIGN in extended char class - /[A-Z]/i did not match LATIN SMALL LETTER LONG S - /[\x{c1}]/i did not match lowercase a-acute Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1 parent 06b0e42 commit 82e7e04

21 files changed

Lines changed: 1814 additions & 46 deletions

dev/architecture/README.md

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# PerlOnJava Architecture Overview
2+
3+
This directory contains architecture documentation for the PerlOnJava compiler and runtime.
4+
5+
## Quick Overview
6+
7+
PerlOnJava is a Perl 5 implementation that compiles Perl source code to JVM bytecode. The system consists of:
8+
9+
1. **Frontend** (`org.perlonjava.frontend`)
10+
- Lexer: Tokenizes Perl source code
11+
- Parser: Builds Abstract Syntax Tree (AST)
12+
- Semantic analysis: Variable resolution, scope handling
13+
14+
2. **Backend** (`org.perlonjava.backend`)
15+
- JVM backend: Emits JVM bytecode using ASM library
16+
- Bytecode interpreter: Interprets a subset of operations for eval STRING
17+
18+
3. **Runtime** (`org.perlonjava.runtime`)
19+
- Runtime types: RuntimeScalar, RuntimeArray, RuntimeHash, RuntimeCode
20+
- Operators: Arithmetic, string, comparison, I/O
21+
- Perl modules: Built-in implementations of core modules
22+
23+
## Key Architecture Documents
24+
25+
| Document | Description |
26+
|----------|-------------|
27+
| [dynamic-scope.md](dynamic-scope.md) | Dynamic scoping via `local` and DynamicVariableManager |
28+
| [lexical-pragmas.md](lexical-pragmas.md) | Lexical warnings, strict, and features |
29+
| [../design/interpreter.md](../design/interpreter.md) | Bytecode interpreter design |
30+
| [../design/variables_and_values.md](../design/variables_and_values.md) | Runtime value representation |
31+
32+
## Compilation Pipeline
33+
34+
```
35+
Perl Source
36+
37+
38+
┌─────────┐
39+
│ Lexer │ Tokenizes source into LexerTokens
40+
└────┬────┘
41+
42+
43+
┌─────────┐
44+
│ Parser │ Builds AST (AbstractNode tree)
45+
└────┬────┘
46+
47+
48+
┌─────────────┐
49+
│ Visitors │ Analysis passes (variable resolution, etc.)
50+
└─────┬───────┘
51+
52+
53+
┌───────────────┐
54+
│ JVM Emitter │ Generates bytecode via ASM
55+
└───────┬───────┘
56+
57+
58+
JVM Bytecode
59+
```
60+
61+
## Runtime Architecture
62+
63+
```
64+
┌────────────────────────────────────────────────┐
65+
│ Perl Code │
66+
└────────────────────────────────────────────────┘
67+
68+
69+
┌────────────────────────────────────────────────┐
70+
│ Runtime Types │
71+
│ RuntimeScalar, RuntimeArray, RuntimeHash │
72+
│ RuntimeCode, RuntimeGlob, RuntimeIO │
73+
└────────────────────────────────────────────────┘
74+
75+
76+
┌────────────────────────────────────────────────┐
77+
│ Global State │
78+
│ GlobalVariable (package variables) │
79+
│ DynamicVariableManager (local scoping) │
80+
│ CallerStack (call frame tracking) │
81+
└────────────────────────────────────────────────┘
82+
83+
84+
┌────────────────────────────────────────────────┐
85+
│ JVM │
86+
└────────────────────────────────────────────────┘
87+
```
88+
89+
## Key Design Decisions
90+
91+
1. **Direct JVM Bytecode**: We emit bytecode directly rather than generating Java source, enabling better optimization and avoiding Java language limitations.
92+
93+
2. **Dual Backend**: JVM bytecode for compiled code, bytecode interpreter for `eval STRING` to avoid runtime class generation overhead.
94+
95+
3. **Dynamic Scoping**: Implemented via `DynamicVariableManager` which maintains a stack of saved values, restored on scope exit.
96+
97+
4. **Lexical Pragmas**: Warnings and strict are tracked in the symbol table at compile time and propagate via `CompilerFlagNode`.
98+
99+
## See Also
100+
101+
- [AGENTS.md](../../AGENTS.md) - Development guidelines
102+
- [dev/design/](../design/) - Detailed design documents

dev/architecture/dynamic-scope.md

Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
# Dynamic Scoping in PerlOnJava
2+
3+
This document explains how PerlOnJava implements Perl's dynamic scoping via the `local` keyword and how the same mechanism is used for other features.
4+
5+
## Overview
6+
7+
Perl's `local` keyword provides dynamic scoping: it temporarily saves a variable's value and restores it when the current scope exits. This is different from lexical scoping (`my`), which creates a new variable visible only in the current block.
8+
9+
```perl
10+
$x = "global";
11+
sub foo {
12+
local $x = "local";
13+
bar(); # sees $x = "local"
14+
}
15+
sub bar {
16+
print $x; # prints "local" when called from foo()
17+
}
18+
foo();
19+
print $x; # prints "global"
20+
```
21+
22+
## Implementation
23+
24+
### Core Components
25+
26+
#### 1. DynamicState Interface
27+
28+
All values that can be dynamically scoped implement `DynamicState`:
29+
30+
```java
31+
public interface DynamicState {
32+
void dynamicSaveState(); // Save current state
33+
void dynamicRestoreState(); // Restore saved state
34+
}
35+
```
36+
37+
Implementations:
38+
- `RuntimeScalar` - scalar variables
39+
- `RuntimeArray` - array variables
40+
- `RuntimeHash` - hash variables
41+
- `RuntimeGlob` - typeglobs
42+
- `DeferBlock` - defer block execution
43+
- `RegexState` - regex match state ($1, $2, etc.)
44+
45+
#### 2. DynamicVariableManager
46+
47+
Manages a stack of saved states:
48+
49+
```java
50+
public class DynamicVariableManager {
51+
private static final Deque<DynamicState> variableStack = new ArrayDeque<>();
52+
53+
// Save current state and push onto stack
54+
public static void pushLocalVariable(DynamicState variable) {
55+
variable.dynamicSaveState();
56+
variableStack.addLast(variable);
57+
}
58+
59+
// Restore all states back to a saved level
60+
public static void popToLocalLevel(int targetLevel) {
61+
while (variableStack.size() > targetLevel) {
62+
DynamicState variable = variableStack.removeLast();
63+
variable.dynamicRestoreState();
64+
}
65+
}
66+
67+
// Get current stack level (saved at block entry)
68+
public static int getLocalLevel() {
69+
return variableStack.size();
70+
}
71+
}
72+
```
73+
74+
#### 3. RuntimeScalar State Management
75+
76+
Each `RuntimeScalar` has its own save stack:
77+
78+
```java
79+
public class RuntimeScalar implements DynamicState {
80+
private static final Stack<RuntimeScalar> dynamicStateStack = new Stack<>();
81+
82+
@Override
83+
public void dynamicSaveState() {
84+
// Save a copy of current state
85+
RuntimeScalar copy = new RuntimeScalar();
86+
copy.type = this.type;
87+
copy.value = this.value;
88+
dynamicStateStack.push(copy);
89+
}
90+
91+
@Override
92+
public void dynamicRestoreState() {
93+
RuntimeScalar saved = dynamicStateStack.pop();
94+
this.type = saved.type;
95+
this.value = saved.value;
96+
}
97+
}
98+
```
99+
100+
### Code Generation
101+
102+
When the compiler sees `local $x`:
103+
104+
1. **Block Entry**: Save current local level
105+
```java
106+
int savedLevel = DynamicVariableManager.getLocalLevel();
107+
```
108+
109+
2. **Local Assignment**: Save and modify variable
110+
```java
111+
DynamicVariableManager.pushLocalVariable(variable);
112+
variable.set(newValue);
113+
```
114+
115+
3. **Block Exit**: Restore all local variables (in finally block)
116+
```java
117+
DynamicVariableManager.popToLocalLevel(savedLevel);
118+
```
119+
120+
### Detection of Local Usage
121+
122+
`FindDeclarationVisitor` scans AST blocks to detect if `local` is used:
123+
124+
```java
125+
public static boolean containsLocalOrDefer(Node blockNode) {
126+
FindDeclarationVisitor visitor = new FindDeclarationVisitor();
127+
visitor.operatorName = "local";
128+
blockNode.accept(visitor);
129+
return visitor.containsLocalOperator || visitor.containsDefer;
130+
}
131+
```
132+
133+
This allows the compiler to skip local setup/teardown for blocks that don't need it.
134+
135+
## Other Uses of DynamicVariableManager
136+
137+
The same mechanism is used for several other features:
138+
139+
### 1. Defer Blocks
140+
141+
`defer { ... }` blocks execute code when scope exits:
142+
143+
```perl
144+
{
145+
defer { print "cleanup\n" }
146+
print "work\n";
147+
} # prints: work, cleanup
148+
```
149+
150+
Implementation:
151+
```java
152+
public class DeferBlock implements DynamicState {
153+
private final RuntimeCode code;
154+
155+
@Override
156+
public void dynamicRestoreState() {
157+
// Execute the defer block
158+
code.apply(new RuntimeArray(), RuntimeContextType.VOID);
159+
}
160+
}
161+
```
162+
163+
### 2. Regex State
164+
165+
Match variables (`$1`, `$2`, `$&`, etc.) are saved/restored:
166+
167+
```java
168+
public class RegexState implements DynamicState {
169+
// Saves: captureGroups, lastMatch, prematch, postmatch, etc.
170+
}
171+
```
172+
173+
This ensures regex state is properly scoped in nested matches.
174+
175+
### 3. Warning Scope (${^WARNING_SCOPE})
176+
177+
Runtime warning suppression uses local semantics:
178+
179+
```perl
180+
{
181+
no warnings 'DateTime'; # Sets local ${^WARNING_SCOPE} = scopeId
182+
DateTime->new(...); # warnif() checks ${^WARNING_SCOPE}
183+
} # ${^WARNING_SCOPE} restored to 0
184+
```
185+
186+
The `CompilerFlagNode` emits:
187+
```java
188+
GlobalRuntimeScalar.makeLocal("${^WARNING_SCOPE}");
189+
scopeVar.set(scopeId);
190+
```
191+
192+
### 4. Signal Handlers
193+
194+
`local $SIG{__WARN__}` and `local $SIG{__DIE__}` use the same mechanism:
195+
196+
```perl
197+
{
198+
local $SIG{__WARN__} = sub { ... };
199+
# warnings go to custom handler
200+
} # original handler restored
201+
```
202+
203+
## Exception Safety
204+
205+
`popToLocalLevel()` is exception-safe:
206+
207+
```java
208+
public static void popToLocalLevel(int targetLevel) {
209+
Throwable pendingException = null;
210+
211+
while (variableStack.size() > targetLevel) {
212+
DynamicState variable = variableStack.removeLast();
213+
try {
214+
variable.dynamicRestoreState();
215+
} catch (Throwable t) {
216+
// Continue cleanup, remember last exception
217+
pendingException = t;
218+
}
219+
}
220+
221+
// Re-throw after all cleanup
222+
if (pendingException != null) {
223+
throw pendingException;
224+
}
225+
}
226+
```
227+
228+
This ensures:
229+
1. All local variables are restored even if one throws
230+
2. Defer blocks all execute even if one throws
231+
3. The last exception "wins" (Perl semantics)
232+
233+
## Performance Considerations
234+
235+
1. **Stack Allocation**: Uses `ArrayDeque` (no synchronization overhead)
236+
2. **Lazy Detection**: `containsLocalOrDefer()` avoids setup for blocks without `local`
237+
3. **Per-Variable Stacks**: Each variable type manages its own save stack
238+
239+
## Files
240+
241+
| File | Purpose |
242+
|------|---------|
243+
| `DynamicState.java` | Interface for saveable state |
244+
| `DynamicVariableManager.java` | Central stack management |
245+
| `RuntimeScalar.java` | Scalar save/restore |
246+
| `RuntimeArray.java` | Array save/restore |
247+
| `RuntimeHash.java` | Hash save/restore |
248+
| `DeferBlock.java` | Defer block execution |
249+
| `RegexState.java` | Regex state save/restore |
250+
| `Local.java` | Code generation helpers |
251+
| `FindDeclarationVisitor.java` | Detection of local usage |
252+
253+
## See Also
254+
255+
- [lexical-pragmas.md](lexical-pragmas.md) - How warnings/strict use this mechanism
256+
- [../design/warnings-scope.md](../design/warnings-scope.md) - Warning scope design

0 commit comments

Comments
 (0)