File read/write of size larger than MPI's 2 GB integer limit

[tpeterka]

The problem is MPI’s use of integers for counts. In this case, MPI_File_write_at() has an integer count, and trying to write more than 2 GB in one write operation produces a corrupted output file, causing it to fail when reading back in. Similar situations exist for other flavors of MPI_File_write and MPI_File_read.

Possible solutions:

• Combine the buffer to be written/read into a smaller number of larger MPI datatypes. The count refers to the number of datatype instances, not total number of bytes.
• Chunk the write/read operation into multiple smaller ones.

The latter solution complicates collective I/O because all processes need to issue the same number of collective calls, which will need to be managed if some operations are chunked at some processes. It will also increase the total number of I/O operations. The former solution is preferable and probably ought to be tried first.