Only use do-end blocks when every trailing keyword is a block keyword - #15725
Merged
josevalim merged 2 commits intoAug 8, 2026
Merged
Conversation
Macro.to_string/1 rendered a trailing keyword list as do-end blocks as
soon as its first key was do:, no matter what the other keys were. The
output parses but means something different:
Macro.to_string(quote do: for(cp <- gc, do: <<cp::utf8>>, into: ""))
for cp <- gc do
<<cp::utf8>>
into
""
end
into is parsed back as a bare variable, so the comprehension no longer
collects into a binary and the code does not compile.
The pre-Code.Normalizer implementation required both that the list start
with do: and that every key be a block keyword. It is still present in
the deprecated Macro.to_string/2 and still gets this right:
defp kw_blocks?([{:do, _} | _] = kw) do
Enum.all?(kw, &match?({x, _} when x in unquote(kw_keywords), &1))
end
Restore the second condition under the same name and guard both paths
into normalize_kw_blocks/4 with it, including the one taken when the AST
already carries do/end metadata. The metadata has to be dropped as well,
since the formatter renders do-end from meta?(meta, :do) alone.
Found by running every .ex and .exs file in this repository through
parse |> Macro.to_string |> parse and comparing the resulting ASTs.
Seven files that previously did not survive the round trip now do,
including lib/elixir/lib/string.ex, and no file regressed.
Closes elixir-lang#15724
Assisted-by: Claude Code:claude-opus-5
Signed-off-by: makoto-developer <72484465+makoto-developer@users.noreply.github.com>
makoto-developer
force-pushed
the
fix-do-block-keywords-v2
branch
from
August 8, 2026 03:53
7ed8834 to
b7fd998
Compare
josevalim
reviewed
Aug 8, 2026
Member
|
💚 💙 💜 💛 ❤️ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #15724
Macro.to_string/1renders a trailing keyword list as do-end blocks as soon as itsfirst key is
do:, regardless of what the remaining keys are. The output parses, butit no longer means the same thing:
intois parsed back as a bare variable, so the comprehension stops collecting into abinary and the code does not compile.
The pre-
Code.Normalizerimplementation guarded on two conditions — the list startswith
do:and every key is a block keyword. It is still there, in the deprecatedMacro.to_string/2(lib/elixir/lib/macro.ex:1534-1541), and it still gets thisright:
Only the first condition was carried over into
Code.Normalizer. This restores thesecond one under the same name, and applies it to both paths into
normalize_kw_blocks/4:Keyword.has_key?(meta, :do)branch, reached when the AST already carriesdo/endmetadata (whatCode.string_to_quoted(token_metadata: true)produces, sotools that parse code and then add an option to a call hit it);
The metadata has to be dropped along with it. The formatter decides on do-end from
meta?(meta, :do)alone (code/formatter.ex:1353), so leaving:do/:endin placewould keep producing blocks no matter what the normalizer decided.
@do_end_keywordsis the same listCode.Formatteruses forcan_force_do_end_blocks?/2(code/formatter.ex:156), andmacro.ex:1535has a thirdcopy. Should these be shared, or is the duplication fine?
elixir_compiler.erl:237compiles
code/formatter.exbeforecode/normalizer.exand the normalizer alreadydepends on
Code.Formatter, so sharing is possible — I did not want to add a publicfunction without asking.
Verification
code_normalizer/quoted_ast_test.exscover bothdirections and both paths: keyword lists with a non-block key stay keyword lists
(including when
do/endare in the metadata), and lists of only block keywordsstill render as do-end. They fail without the change.
make test— elixir 7503, ex_unit 465, logger 165, eex 118,iex 289, mix 941.
.ex/.exsfile underlib/*/{lib,test}(467 files, excluding mix test fixtures) throughparse |> Macro.to_string |> parseand comparing ASTs: 63 files failed before,56 after. The 7 fixed are
lib/elixir/lib/string.exand six Mix internals(
mix/state.ex,mix/project.ex,mix/dep.ex,mix/tasks/xref.ex,mix/compilers/protocol.ex,mix/compilers/test.ex). One further file,kernel/expansion_test.exs, goes from unparseable to parseable. No fileregressed. The remaining 56 are unrelated round-trip differences.
Notes
mix formatis unaffected —Code.format_string!/2does not go throughCode.Normalizer. The blast radius isMacro.to_string/1andCode.quoted_to_algebra/2.elixir_tokenizer.erl:1681-1684tokenizes exactly
after/else/catch/rescueas block keywords, so a list thatshould render as do-end cannot fall through to the keyword-list form.
for x <- y, do: x, into: ""keeps the original keyorder even though
into:beforedo:is the more idiomatic spelling. Reorderingkeys is out of scope here.
dobody and a non-block key, the keyword-list form isverbose:
code.ex:759-764documents:force_do_end_blocksas converting "all keywords" intodo-end blocks. That was already inaccurate for
Code.format_string!/2; after thischange it is also inaccurate for
Code.quoted_to_algebra/2(
Code.quoted_to_algebra({:foo, [], [[do: 1, bar: 2]]}, force_do_end_blocks: true)stays a keyword list). Happy to fix the doc here or in a separate PR.
Macro.to_string/1output (09d43562eFix edge case in normalizer for keyword args #13924,bc50d9494Fix Code.Normalizer for keyword operand with :do key #13250,673fe4a89Fix crash in Macro.to_string #13905,
e54b87c18Fix formatter adding extra escapes to remote call functions #13960,5b8b8e358Reintroduce escaped trailing newlines in heredocs in Macro.to_string/2 #15355) all shipped without one, and the oldoutput here does not parse, so nothing could have depended on it.
Assisted-by: Claude Code:claude-opus-5