packing result is corrupted very rarely

[takaaki7]

We are using message pack to serialize k/v structure data and save in to db. In very rare case(about 0.000000001%), the result is corrupted.

We serializes a lot of data with thread local MessageBufferPacker, and 160 threads may executes serialization concurrently.
Seeing serializing result, in the middle of the some value, a part of the other data is mixed in. (no exception is occurred)

I've not yet to figure out how to reproduce, and cannot share the actual data here because it is production data.

Serialization input / output is similar to following. (actual data is more large(above 10kb, more fields))

input

data A: {field1: 'value1', field2:'value2', field3:'value3', field4:'value4'}
data B: {field6: 'value6', field7:'value7'}

output of data A

[map(4)][string(5)]field1[string(5)]valvalue6[string(5)]field7[string(5)]value7[string(5)]field4[string(5)]value4
(label [ ] means field type and value length):

data B is mixed in from the middle of value2
Because the real string length is different from length defined in header, cannot deserialize the result (causes error).

I think possible causes is

• Previous packing state is remained illegally. (MessageBufferPacker.clear() is insufficient to clear data)
• Unsafe writes data into illegal address.

Is this known behavior? Do you have any idea?

Code:

public static final byte[] VERSION = new byte[] {1};

public class MessagePackSerialization  {
  private final ThreadLocal<MessageBufferPacker> messagePacker =
      ThreadLocal.withInitial(MessagePack::newDefaultBufferPacker);
      
      
  public ByteBuffer serialize(JsonNode objectNode) {
    MessageBufferPacker packer = messagePacker.get();
    packer.clear();
    try {
      packer.writePayload(VERSION);
      packJsonNode(packer, objectNode);
      return packer.toMessageBuffer().sliceAsByteBuffer();
    } catch (IOException e) {
      throw new RuntimeException("failed to serialize:" + objectNode, e);
    }
  }

  private static void packJsonNode(MessageBufferPacker packer, JsonNode value) throws IOException {
    JsonNodeType nodeType = value.getNodeType();
    switch (nodeType) {
      case MISSING:
      case NULL:
        packer.packNil();
        return;
      case BOOLEAN:
        packer.packBoolean(value.booleanValue());
        return;
      case NUMBER:
        JsonParser.NumberType numberType = value.numberType();
        switch (numberType) {
          case INT:
            if (value.isShort()) {
              packer.packShort(value.shortValue());
            } else {
              packer.packInt(value.intValue());
            }
            return;
          case LONG:
            packer.packLong(value.longValue());
            return;
          case BIG_INTEGER:
            packer.packDouble(value.bigIntegerValue().doubleValue());
            return;
          case FLOAT:
            packer.packFloat(value.floatValue());
            return;
          case DOUBLE:
            packer.packDouble(value.doubleValue());
            return;
          default:
            throw new IllegalArgumentException("unsupported type:" + numberType + ", " + value);
        }
      case STRING:
        packer.packString(value.textValue());
        return;
      case OBJECT:
        packer.packMapHeader(value.size());
        Iterator<Map.Entry<String, JsonNode>> fields = value.fields();
        while (fields.hasNext()) {
          var field = fields.next();
          packer.packString(field.getKey());
          packJsonNode(packer, field.getValue());
        }
        return;
      case ARRAY:
        packer.packArrayHeader(value.size());
        for (JsonNode v : value) {
          packJsonNode(packer, v);
        }
        return;
      default:
        throw new IllegalArgumentException("unsupported type:" + nodeType + ", " + value);
    }
  }
}

environment:
openjdk-17
'org.msgpack:msgpack-core:0.8.20'
'com.fasterxml.jackson.core:jackson-core:2.10.0'