[java-firestore] BulkWriter: a write that throws synchronously corrupts its batch, so a later write and flush() never complete #14631

Issue Details

When set(), create() or update() on a BulkWriter throws synchronously (for example because CustomClassMapper cannot encode a value), the operation has already been added to the current batch without a write. When that batch commits, the results are matched to operations out of step: a later write in the same batch is applied to the database but its future never completes, and neither does flush().

sendOperationLocked calls bulkCommitBatch.enqueueOperation(op) before enqueueOperationOnBatchCallback.apply(bulkCommitBatch):

apply encodes the value and throws out of set(), leaving the operation in BulkCommitBatch.pendingOperations with no entry in the batch's writes. bulkCommit() pairs writeResults.get(i) with pendingOperations.get(i), so from the refused operation onward each result goes to the operation before its own, and the last operation never receives one. executeWrite also returns before attaching the handlers that decrement pendingOpsCount, so the refused operation's slot leaks too.

Rolling back on failure (removing the operation from the batch when apply throws, and releasing its pending slot) would keep the two lists aligned. enqueueOperation cannot simply move after apply, because addWrite calls wrapResult(writeIndex), which reads pendingOperations.

Found while building an Apache Flink sink on BulkWriter (flink-gcp/flink-connector-gcp), which currently works around it.

Environment

• OS Type and Version: macOS 26 (arm64)
• Java Version and JDK Vendor: Eclipse Temurin 17.0.20

The reproducer runs against the Firestore emulator (gcr.io/google.com/cloudsdktool/google-cloud-cli:587.0.0-emulators). The defect is in client-side logic, so no GCP project is involved.

Dependencies

• Libraries-Bom: 26.90.0
  • google-cloud-firestore 3.49.0
• Gax: 2.87.0
• Auth: 1.54.0

The affected code is identical in 3.49.0 and on main.

Reproducer

Run with FIRESTORE_EMULATOR_HOST pointing at the emulator, e.g. docker run -p 8080:8080 gcr.io/google.com/cloudsdktool/google-cloud-cli:587.0.0-emulators gcloud emulators firestore start --host-port=0.0.0.0:8080.

package repro;

import com.google.api.core.ApiFuture;
import com.google.cloud.firestore.BulkWriter;
import com.google.cloud.firestore.DocumentReference;
import com.google.cloud.firestore.Firestore;
import com.google.cloud.firestore.FirestoreOptions;
import com.google.cloud.firestore.WriteResult;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

/** Run with FIRESTORE_EMULATOR_HOST set. */
public class SynchronousRefusalCorruptsBatch {
  public static void main(String[] args) throws Exception {
    Firestore db = FirestoreOptions.newBuilder().setProjectId("demo").build().getService();
    String collection = "refusal-" + System.nanoTime();
    BulkWriter bulkWriter = db.bulkWriter();

    ApiFuture<WriteResult> before = bulkWriter.set(db.document(collection + "/before"), Map.of("v", 1));
    try {
      // CustomClassMapper does not support Short, so set() throws synchronously.
      bulkWriter.set(db.document(collection + "/refused"), Map.of("v", (short) 1));
    } catch (IllegalArgumentException e) {
      System.out.println("set(refused) threw: " + e);
    }
    DocumentReference afterDoc = db.document(collection + "/after");
    ApiFuture<WriteResult> after = bulkWriter.set(afterDoc, Map.of("v", 1));
    ApiFuture<Void> flush = bulkWriter.flush();

    System.out.println("before completed: " + before.get(10, TimeUnit.SECONDS));
    try {
      System.out.println("after completed: " + after.get(10, TimeUnit.SECONDS));
    } catch (TimeoutException e) {
      System.out.println("after did not complete within 10 s");
    }
    System.out.println("after exists in the database: " + afterDoc.get().get().exists());
    try {
      flush.get(10, TimeUnit.SECONDS);
      System.out.println("flush() completed");
    } catch (TimeoutException e) {
      System.out.println("flush() did not complete within 10 s");
    }
    System.exit(0);
  }
}

Logs and Stack Trace

set(refused) threw: java.lang.IllegalArgumentException: Could not serialize object. Numbers of type Short are not supported, please use an int, long, float, double or BigDecimal
before completed: com.google.cloud.firestore.WriteResult@fd49d5c7
after did not complete within 10 s
after exists in the database: true
flush() did not complete within 10 s

Behavior

Deterministic. The caller sees the IllegalArgumentException and may reasonably continue with the next write, but the BulkWriter instance is already corrupt: flush() and close() no longer complete.