Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 76 additions & 9 deletions dev/design/feature-audit-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,74 @@ logs are in `/tmp/feature-audit-*` and are not committed by default.
| `postderef_qq` interpolation | pass | pass | pass | Existing `postderef_interpolation.t` passes all four assertions. |
| `unicode_eval` / `evalbytes` | pass | pass | pass | Existing `evalbytes.t` passes all three assertions. |
| Container `refaliasing` | pass | pass | pass | Existing `refaliasing_containers.t` passes both aliasing assertions. |
| `Safe` sandbox | pass | pass | pass | `safe_permit_only.t` passes all five assertions. |
| `encoding` pragma | pass | pass | pass | `encoding_pragma.t` and the encoding-layer probe pass. |
| `integer` pragma | pass | pass | pass | Native-width arithmetic and bitwise tests pass. |
| `Unicode::Normalize` | pass | pass | pass | Canonical and compatibility normalization tests pass. |
| `builtin::export_lexically` | pass | pass | pass | Existing lexical export test passes all assertions. |
| Overload `++`, `.`, and `=` | pass | pass | pass | Focused increment, concatenation, and copy-constructor tests pass. |
| `overloading` pragma | pass | pass | pass | Existing lexical enable/disable test passes. |
| `bigint` / `bignum` | pass | pass | precision failure / timeout | Numeric pragmas are partial; `bigrat` passes separately. |
| Local glob/filehandle boundaries | pass | pass | pass | Four focused glob/typeglob tests pass; exact `local *HANDLE = *HANDLE` semantics remain separate. |
| `unicode_strings` | pass | pass | pass | Unicode uppercase mapping for a non-UTF-8 byte string passes. |
| `bigrat` | pass | pass | pass | Isolated rational-arithmetic probe passes. |
| Multibyte encoded I/O positioning | pass | pass | pass | Representative `seek`, `tell`, and `truncate` behavior passes. |

The exact commands and test names are documented in the probe README. A
passing probe does not promote a broad feature automatically: semantic and
edge-case coverage must be sufficient for the documented claim.

## Completion Inventory

The remaining matrix entries have been reconciled into the following evidence
classes. An item is considered audited when its status is supported by a
probe, an implementation/design source, or an explicit JVM boundary—not when
the feature itself is implemented.

### Probe-confirmed remaining gaps

| Gap | Evidence | Disposition |
|---|---|---|
| Restricted hashes | [`remaining_semantics.t`](../tools/feature-audit/remaining_semantics.t) | `Hash::Util` loads, but lock enforcement fails on both backends. |
| `fork` | [`remaining_semantics.t`](../tools/feature-audit/remaining_semantics.t) | Native Perl returns a child PID; both backends leave it undefined. |
| DBM | [`remaining_semantics.t`](../tools/feature-audit/remaining_semantics.t) | Native round trip passes; `dbmopen` is explicitly unimplemented on both backends. |
| `ops` | [`remaining_semantics.t`](../tools/feature-audit/remaining_semantics.t) | Native module loads; PerlOnJava cannot locate `ops.pm`. |
| Interpreter `bigint`/`bignum` | [`numeric_bigint.t`](../tools/feature-audit/numeric_bigint.t), [`numeric_bignum.t`](../tools/feature-audit/numeric_bignum.t) | Interpreter loses `bigint` precision and times out on `bignum`; JVM and native checks pass. |

### Passing representative behavior

`unicode_strings`, `bigrat`, encoded `seek`/`tell`/`truncate`, `Safe`,
`encoding`, `integer`, `Unicode::Normalize`, lexical builtin export,
`overloading`, local glob/filehandle boundaries, selected overload operators,
taint mode, bytes scope, POSIX math, warnings bits, and regex conditional/
Unicode cases have native/JVM/interpreter evidence. Their remaining matrix
notes describe only the untested edge cases or broader APIs.

### Source- or boundary-confirmed gaps

These do not have a meaningful “system Perl versus jperl” probe because the
gap is an embedding interface, debugger command, native boundary, or JVM
execution constraint:

- debugger custom modules, `perl5db.pl`, restart, history, command editing,
conditional breakpoints, watches, and line actions — tracked in the
[debugger reference](../../docs/reference/debugger.md);
- missing command-line switches, startup `sitecustomize.pl`, compiler hint
variables, lexical `no strict refs`, `locale`, `ops`, `vmsish`, builtin
`load_module`, exporter `*glob`, and the remaining overload operators —
tracked in the [feature matrix](../../docs/reference/feature-matrix.md);
- direct Perl-to-Java interoperation and native XS/C loading — explicitly
bounded in the [feature matrix](../../docs/reference/feature-matrix.md);
- JSR-223 `Invocable`, `ScriptContext` writers, Java `Bindings`, and the
`THREADING` factory parameter — documented in the [JSR-223 design](jsr223-perlonjava-web.md);
- true OS-level `fork`, native Perl `dump`, DBM, class-name invocation, and
JVM-dependent destructor/file lifecycle behavior — documented as JVM
boundaries in the [feature matrix](../../docs/reference/feature-matrix.md).

These entries are not left as unexplained “TODO” claims: each has a current
status, an evidence source, and—where applicable—a separate implementation
follow-up rather than an audit ambiguity.

## Audit Method

### 1. Build the inventory
Expand Down Expand Up @@ -194,7 +257,7 @@ documentation-only planning change.

## Progress Tracking

### Current Status: Initial probe batch complete; broader audit in progress
### Current Status: Documented gap inventory audited; implementation follow-ups remain

### Completed Phases

Expand All @@ -212,17 +275,21 @@ documentation-only planning change.
and container `refaliasing`; corrected their stale matrix status.
- [x] Validated the full 11-field `caller()` tuple and key subroutine metadata
with a dedicated native/JVM/interpreter probe.
- [x] Audited the main pragma, taint, Safe, overload, bytes, I/O, POSIX,
warnings, glob, and normalization test batch on all three runtimes.
- [x] Audited small `bigint`/`bignum` probes; recorded the interpreter
precision failure and timeout as partial-support evidence.
- [x] Reconciled the remaining matrix entries into probe-confirmed,
representative-pass, and source/JVM-boundary evidence classes.

### Next Steps

1. Add probes for the remaining language, pragma, overload, regex, debugger,
module, XS, and JSR-223 gaps.
2. Validate full `caller` fields and multibyte `seek`/`tell`/`truncate`
semantics rather than only their basic cases.
3. Reconcile remaining stale and duplicate documentation claims.
4. Update related module and design pages from confirmed evidence.
5. Route genuine backend regressions, such as interpreter fd retention, to
implementation-specific follow-up work.
1. Route confirmed implementation gaps to code-fix work, starting with
interpreter numeric precision and fd retention.
2. Extend representative probes when a future implementation changes a
partial status or when a module’s broader API is promoted.
3. Keep the matrix and related design/module pages synchronized with future
audit evidence.

### Open Questions and Blockers

Expand Down
5 changes: 5 additions & 0 deletions dev/tools/feature-audit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@ Perl version and platform when behavior is version- or platform-dependent.
| `remaining_semantics.t` | Focused checks for smartmatch, restricted hashes, overload, `caller`, `ops`, DBM, `fork`, `dump`, and multibyte file positioning |
| `regex_remaining.t` | Executable conditional expressions, grapheme clusters, and `Extended_Pictographic` |
| `caller_fields.t` | Full 11-field `caller()` tuple and subroutine metadata |
| `numeric_bigint.t` | BigInt precision behavior |
| `numeric_bignum.t` | BigNum arbitrary-precision division |
| `numeric_bigrat.t` | BigRat rational arithmetic |
| `unicode_strings.t` | Unicode case mapping for non-UTF-8 byte strings |
| `multibyte_io.t` | Representative encoded-handle `seek`, `tell`, and `truncate` behavior |

## Interpreting results

Expand Down
14 changes: 14 additions & 0 deletions dev/tools/feature-audit/multibyte_io.t
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
use strict;
use warnings;
use File::Temp qw(tempfile);
use Test::More tests => 5;

my ($fh, $path) = tempfile(UNLINK => 1);
binmode $fh, ':encoding(UTF-8)';
print {$fh} "éx" or die "write: $!";

is(tell($fh), 5, 'tell reports encoded byte position');
ok(seek($fh, 2, 0), 'seek to an encoded byte position succeeds');
is(tell($fh), 2, 'tell reports the seeked encoded position');
is(unpack('H*', <$fh> // ''), 'a978', 'read after multibyte seek is stable');
ok(truncate($fh, 1), 'truncate on an encoded handle succeeds');
6 changes: 6 additions & 0 deletions dev/tools/feature-audit/numeric_bigint.t
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
use strict;
use warnings;
use Test::More tests => 1;
use bigint;

is(2 ** 64 + 1, 18446744073709551617, 'bigint preserves integer precision');
6 changes: 6 additions & 0 deletions dev/tools/feature-audit/numeric_bignum.t
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
use strict;
use warnings;
use Test::More tests => 1;
use bignum;

ok((1 / 3) > 0 && (1 / 3) < 1, 'bignum supports arbitrary-precision division');
6 changes: 6 additions & 0 deletions dev/tools/feature-audit/numeric_bigrat.t
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
use strict;
use warnings;
use Test::More tests => 1;
use bigrat;

is(1 / 3 + 1 / 3 + 1 / 3, 1, 'bigrat preserves rational arithmetic');
9 changes: 9 additions & 0 deletions dev/tools/feature-audit/unicode_strings.t
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
use strict;
use warnings;
use Test::More tests => 2;

use feature 'unicode_strings';

my $byte = "\xDF";
is(unpack('H*', lc $byte), 'df', 'unicode_strings preserves lowercase byte behavior');
is(unpack('H*', uc $byte), '5353', 'unicode_strings applies Unicode uppercase mapping');
2 changes: 1 addition & 1 deletion docs/about/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Work currently in progress:

- **Warnings Subsystem** — Improving lexical `warnings` pragma scope handling and warning message formatting for Perl5 compatibility. See `dev/design/warnings-scope.md`.
- **Regex Improvements** — Ongoing compatibility fixes for regex edge cases, POSIX character classes, and Unicode properties.
- **Overload Completeness** — Adding missing overload operators: `++`, `--`, `=` (copy constructor), bitwise, string repeat, concatenation. See [Feature Matrix — overload](../reference/feature-matrix.md#pragmas).
- **Overload Completeness** — Adding remaining overload operators: `--`, bitwise, string repeat, and their compound forms. `++`, copy-constructor `=`, and concatenation are verified. See [Feature Matrix — overload](../reference/feature-matrix.md#pragmas).
- **`caller` Extended Information** — Implementing `(caller($level))[3..11]` for subroutine names, `wantarray`, `evaltext`, hints. Required for better error messages and Carp compatibility.
- **Compiler Hardening** — Automatic fallback to interpreter mode when JVM "Method too large" errors occur. Fix remaining global variable aliasing edge cases in `for` loops.
- **perl5 Test Suite** — Expanding pass rates across `perl5_t/t/` categories (op, re, uni, mro, io, lib).
Expand Down
25 changes: 13 additions & 12 deletions docs/reference/feature-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -567,7 +567,7 @@ my @copy = @{$z}; # ERROR
- `:unix` - Unix-style line endings (LF only)
- `:encoding(ENCODING)` - Specific character encoding
- ✅ **Layer stacking**: Multiple layers can be combined (e.g., `:raw:utf8`)
- **Multibyte encoding support for `seek`, `tell`, `truncate`**: These operations are not yet implemented for multibyte encodings.
- **Multibyte encoding support for `seek`, `tell`, `truncate`**: Representative encoded-handle positioning and truncation pass on both backends. See the [audit probe](../../dev/tools/feature-audit/multibyte_io.t); additional platform/encoding edge cases remain suitable for follow-up coverage.

### Supported Encodings
The `:encoding()` layer supports all encodings provided by Java's `Charset.forName()` method:
Expand Down Expand Up @@ -649,20 +649,22 @@ The `:encoding()` layer supports all encodings provided by Java's `Charset.forNa
- 🚧 **bytes** pragma
- 🚧 **feature** pragma
- ✅ Features implemented: `fc`, `say`, `current_sub`, `isa`, `state`, `try`, `defer`, `bitwise`, `postderef`, `postderef_qq`, `evalbytes`, `unicode_eval`, `refaliasing`, `module_true`, `signatures`, `class`, `keyword_all`, `keyword_any`.
- ❌ Feature still missing: `unicode_strings`.
- ✅ `unicode_strings` (see the [audit probe](../../dev/tools/feature-audit/unicode_strings.t)).
- 🚧 **warnings** pragma
- 🚧 **attributes** pragma: `MODIFY_*_ATTRIBUTES`/`FETCH_*_ATTRIBUTES` callbacks for subroutines and variables.
- ❌ **bignum, bigint, and bigrat** pragmas
- ❌ **encoding** pragma
- ❌ **integer** pragma
- 🚧 **bignum** and **bigint** pragmas: basic checks pass on the JVM backend; the interpreter loses `bigint` precision and does not complete the basic `bignum` probe within the audit timeout. See the [bignum](../../dev/tools/feature-audit/numeric_bignum.t) and [bigint](../../dev/tools/feature-audit/numeric_bigint.t) probes.
- ✅ **bigrat** pragma: isolated rational-arithmetic probe passes on all backends; see the [audit probe](../../dev/tools/feature-audit/numeric_bigrat.t).
- ✅ **encoding** pragma: the supported encoding pragma forms pass the native/JVM/interpreter audit batch.
- ✅ **integer** pragma: native-width arithmetic and bitwise behavior pass the native/JVM/interpreter audit batch.
- ❌ **locale** pragma
- ❌ **ops** pragma
- 🚧 **re** pragma for regular expression options: Implemented `is_regexp`.
- 🚧 **vmsish** pragma.
- ✅ **subs** pragma.
- 🚧 **builtin** pragma:
- ✅ Implemented: `true` `false` `is_bool` `inf` `nan` `weaken` `unweaken` `is_weak` `blessed` `refaddr` `reftype` `created_as_string` `created_as_number` `stringify` `ceil` `floor` `indexed` `trim` `is_tainted`.
- ❌ Missing: `export_lexically`, `load_module`
- ✅ `export_lexically`.
- ❌ Missing: `load_module`
- 🚧 **overload** pragma:
- ✅ Implemented: `""`, `0+`, `bool`, `fallback`, `nomethod`.
- ✅ Implemented: `!`, `+`, `-`, `*`, `/`, `%`, `int`, `neg`, `log`, `sqrt`, `cos`, `sin`, `exp`, `abs`, `atan2`, `**`.
Expand All @@ -671,12 +673,11 @@ The `:encoding()` layer supports all encodings provided by Java's `Charset.forNa
- ✅ Implemented: `qr`.
- ✅ Implemented: `+=`, `-=`, `*=`, `/=`, `%=`.
- ✅ Implemented: `<>`.
- ❌ Missing: `++`, `--`, `=`.
- ❌ Missing: `&`, `|`, `^`, `~`, `<<`, `>>`, `&.`, `|.`, `^.`, `~.`, `x`, `.`.
- `++`, `.`, and `=` copy-constructor behavior pass focused audit tests.
- ❌ Missing: `--`, `&`, `|`, `^`, `~`, `<<`, `>>`, `&.`, `|.`, `^.`, `~.`, `x`.
- ❌ Missing: `**=`, `<<=`, `>>=`, `x=`, `.=`, `&=`, `|=`, `^=`, `&.=`, `|.=`, `^.=`.
- ❌ Missing: `-X`.
- ❌ Missing: `=` copy constructor for mutators.
- ❌ **overloading** pragma
- ✅ **overloading** pragma: lexical enable/disable behavior passes the focused audit batch.



Expand Down Expand Up @@ -760,14 +761,14 @@ The `:encoding()` layer supports all encodings provided by Java's `Charset.forNa
- 🚧 **DynaLoader** placeholder module.
- 🚧 **HTTP::Tiny** some features untested: proxy settings.
- 🚧 **POSIX** module.
- 🚧 **Unicode::Normalize** `normalize`, `NFC`, `NFD`, `NFKC`, `NFKD`.
- **Unicode::Normalize**: canonical and compatibility normalization passes the focused audit batch.
- ✅ **Archive::Tar** module.
- ✅ **Archive::Zip** module.
- ✅ **IPC::Open2** module.
- ✅ **IPC::Open3** module.
- ✅ **Net::FTP** module.
- ✅ **Net::Cmd** module.
- **Safe** module.
- **Safe** module: permit-only and default sandbox behavior passes the focused audit batch.

### Non-core modules
- 🟡 **Object::Pad**: core class, field, method, parameter, and inheritance
Expand Down
Loading