docs: document why json_dumps requires sorted keys for cache hashing

Add comprehensive docstring to json_dumps() explaining that key sorting
is critical for deterministic JSON output, which is used to compute
cache hashes in incremental type checking. Without sorted keys,
dictionaries with the same content but different key insertion order
would produce different hashes, causing incorrect cache invalidation.

This addresses the TODO comment and clarifies the relationship between
JSON serialization and the caching system.

Author: bohdansolovie

mypy/util.py

@@ -925,12 +925,23 @@ def quote_docstring(docstr: str) -> str:
 
 
 def json_dumps(obj: object, debug: bool = False) -> bytes:
+    """Serialize an object to JSON bytes.
+    
+    Keys are always sorted to ensure deterministic output. This is critical for
+    incremental type checking: the JSON output is used to compute cache hashes
+    (via hash_digest/hash_digest_bytes in build.py). Without key sorting, dictionaries
+    with the same content but different key insertion order would produce different
+    JSON strings, leading to different hashes and incorrect cache invalidation.
+    
+    For example, in testIncrementalInternalScramble, the test verifies that mypy
+    correctly handles incremental updates. If keys weren't sorted, the cache hash
+    would change even when the actual content is the same, causing unnecessary
+    re-checking and test failures.
+    """
     if orjson is not None:
         if debug:
             dumps_option = orjson.OPT_INDENT_2 | orjson.OPT_SORT_KEYS
         else:
-            # TODO: If we don't sort keys here, testIncrementalInternalScramble fails
-            # We should document exactly what is going on there
             dumps_option = orjson.OPT_SORT_KEYS
 
         try:
@@ -942,7 +953,6 @@ def json_dumps(obj: object, debug: bool = False) -> bytes:
     if debug:
         return json.dumps(obj, indent=2, sort_keys=True).encode("utf-8")
     else:
-        # See above for sort_keys comment
         return json.dumps(obj, sort_keys=True, separators=(",", ":")).encode("utf-8")