MDEV-39091 BACKUP SERVER of ENGINE=RocksDB to the local file system

[Thirunarayanan]

Wire the RocksDB (MyRocks) engine into BACKUP SERVER TO and BACKUP SERVER WITH (streaming). RocksDB SST files are immutable, so a consistent point-in-time snapshot is obtained cheaply with a rocksdb::Checkpoint. The checkpoint is frozen at BACKUP_PHASE_NO_COMMIT because the checkpoint files are immutable, the actual copy is deferred to BACKUP_PHASE_FINISH, after the backup MDL has been released, and the server-side checkpoint is removed at end of FINISH. No user-level lock is needed: MDL_BACKUP_START already serializes concurrent BACKUP SERVER.

Files are copied into the target's "#rocksdb" subdirectory. The default rocksdb_datadir ("./#rocksdb") makes restore transparent: on restore the files land at /#rocksdb, so pointing a server at the extracted backup just works, with no copy-back and no RocksDB prepare step.

storage/rocksdb/rdb_backup_server.{h,cc}:
New RocksDB_backup context and the three handlerton hooks.
The checkpoint file list is drained across N CONCURRENT step threads via a lock-free atomic cursor; each file is copied with copy_entire_file() (directory target) or backup_stream_*(). Since SSTs are immutable, the sendfile(2) fast path (backup_stream_append_async) is used for the stream target.

rdb_create_checkpoint(): Factor the checkpoint creation
rdb_remove_checkpoint(): Remove the checkpoint creation logic.
rdb_get_datadir(): Get the rocksdb data directory.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[gemini-code-assist]

Code Review

This pull request implements BACKUP SERVER support for the RocksDB (MyRocks) storage engine, introducing new backup hooks to create consistent checkpoints and copy or stream the immutable checkpoint files. The review feedback highlights several critical issues: a potential security and correctness risk where an empty datadir could write to the root directory, potential backup corruption due to an unrestored file offset after calling lseek, a resource leak on backup abort because cleanup is only executed during the finish phase, and a potential crash on Windows if target.path is null.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

If rdb_get_datadir() returns nullptr or an empty string, dir will resolve to / after appending the slash, causing the checkpoint directory to be created at /mariabackup-checkpoint. This is a security and correctness risk as it attempts to write to the root of the filesystem. Defaulting to the current directory . when the datadir is empty or null is much safer.

  const char *datadir= rdb_get_datadir();
  if (!datadir || !*datadir)
    datadir= ".";
  std::string dir(datadir);

[dr-m]

rocksdb_datadir can never be nullptr. Elsewhere, it is assumed that it cannot refer to an empty string either, even though this might not be enforced:

std::string rdb_corruption_marker_file_name() {
  std::string ret(rocksdb_datadir);
  ret.append("/ROCKSDB_CORRUPTED");
  return ret;
}

[gemini-code-assist]

Using lseek(src, 0, SEEK_END) to determine the file size leaves the file offset at the end of the file. If the underlying implementation of backup_stream_append_async (or any fallback path) relies on the current file offset (e.g., using read() instead of pread()), it will read 0 bytes, leading to silent backup corruption. The file offset must be restored to the beginning of the file using lseek(src, 0, SEEK_SET) before proceeding.

    uint64_t end= uint64_t(lseek(src, 0, SEEK_END));
    if (lseek(src, 0, SEEK_SET) == (off_t)-1 ||
        backup_stream_start(sink.stream, rel.c_str(), 0644, end, nullptr, 0) ||
        backup_stream_append_async(src, sink.stream, 0, end))

[gemini-code-assist]

If the backup is aborted, rocksdb_backup_end is called with BACKUP_PHASE_ABORT. Currently, the cleanup logic (removing the checkpoint and deleting the bk context) is only executed when phase == BACKUP_PHASE_FINISH. This causes a memory leak and leaves stale checkpoint directories on disk when a backup fails or is aborted. The cleanup should be performed on both BACKUP_PHASE_FINISH and BACKUP_PHASE_ABORT.

  if (phase == BACKUP_PHASE_FINISH || phase == BACKUP_PHASE_ABORT)
  {
    bk->remove_checkpoint();
    delete bk;
  }

[gemini-code-assist]

On Windows, if target.path is nullptr, constructing std::string dest_path(target.path) results in undefined behavior (typically a crash). Adding a null check prevents potential crashes.

  if (sink.stream == sink.NO_STREAM)
  {
    /* Directory target */
    if (!target.path)
    {
      my_error(ER_CANT_CREATE_FILE, MYF(0), rel.c_str(), EINVAL);
      return 1;
    }
    std::string dest_path(target.path);

[dr-m]

This looks mostly OK to me. The interesting part is RocksDB_backup::copy_one(), which copies or streams a file.

We are lacking some test coverage.

[dr-m]

This is failing on Windows:

backup.backup_rocksdb                    w8 [ fail ]
        Test ended at 2026-07-15 12:53:53
CURRENT_TEST: backup.backup_rocksdb
mysqltest: At line 47: query 'BACKUP SERVER WITH 2 CONCURRENT '$script'' failed: ER_IO_WRITE_ERROR (1811): IO Write error: (2, No such file or directory) BACKUP SERVER

Curiously, for the CMAKE_BUILD_TYPE=Debug build on Windows, this test is being skipped:

backup.backup_rocksdb                    w15 [ skipped ]  Test requires MyRocks engine

[dr-m]

Is this really valid on Microsoft Windows? The test backup.backup_stream is actually executing a script directly with cmd.exe, using only the additional dependency on cat:

--remove_files_wildcard $MYSQL_TMP_DIR/. stream.bat
--write_file $MYSQL_TMP_DIR/stream.bat
#!/bin/sh
exec cat > $MYSQL_TMP_DIR/$1.tar
EOF

--let $script=$MYSQL_TMP_DIR/stream.bat
if (!$MARIADB_UPGRADE_EXE) {
# Because we may run on Linux /dev/shm which may be mounted as noexec,
# we cannot rely on chmod +x, but must explicitly invoke a shell on the script.
--let $script=/bin/sh $script
}
if ($MARIADB_UPGRADE_EXE)
{
--exec echo "@cat > %MYSQL_TMP_DIR%\%1.tar" > $MYSQL_TMP_DIR/stream.bat
}

It would be useful to preserve the comment regarding a possible noexec mount option. The #!/bin/sh line is actually redundant and should be removed in #4817.

[dr-m]

It would good to have two separate tests: non-streaming and streaming, say, backup.backup_rocksdb and backup.backup_rocksdb_stream.

Other tests for this storage engine live in storage/rocksdb/mysql-test. I was wondering if these tests should be moved there as well. Then I realized that we already have some ENGINE=RocksDB specific tests outside that subdirectory, in mysql-test/suite/mariabackup. That reminds me that we’re missing any tests to cover the mariadb-backup wrapper script with ENGINE=RocksDB. Even in the case that the wrapper is not supposed to support this storage engine, it should fail with a clear error message, which needs to be tested.

Side note: The tests under storage/rocksdb/mysql-test are not fully covered. I see that amd64-windows-packages only runs the rocksdb suite, not rocksdb_rpl or rocksb_hotbackup. The latter one is not even being run on amd64-debian-12-rocksdb.

[dr-m]

If the streaming tools are missing, we’re skipping all test coverage, even though it is technically possible to cover backup to a mounted file system.

[dr-m]

One pointer is being tested for null-ness with unary !, another with a binary ==. That is inconsistent.

[dr-m]

This is duplicating a declaration in storage/maria/ma_backup_server.cc. I wonder if we should declare such a buffer in sql/sql_backup.cc. InnoDB uses a much larger field_ref_zero, which needs to be allocated at runtime because the needed large alignment cannot be satisfied by some linkers.

[dr-m]

Starting with C++98, the correct way to declare a parameter unused is to omit its name, that is, just write the type name THD*,.

[dr-m]

This can be simplified:

  if (phase != BACKUP_PHASE_FINISH)
    return 0;
  RocksDB_backup *bk= static_cast<RocksDB_backup *>(sink->ha_data);
  if (!bk)
    return 0;
  size_t i= bk->claim_next();

[dr-m]

We should check the parameter phase before dereferencing sink.

[dr-m]

There is no mention that this will be populated by scan_checkpoint() and will remain constant after that.