-
Notifications
You must be signed in to change notification settings - Fork 7
Buffers
Code executed on the GPU works with memory physically located on the video card, so it is necessary to be able to allocate arrays in that memory and move data between device and host memory.
To make this more convenient, this library provides a generic interface based on the lisp array API. It is also implemented for lisp arrays themselves.
- (bufferp object) → nil / :lisp / :foreign
This method can be used to check if the object implements the interface.
For supported objects it returns either :lisp or :foreign; this specifies which of the element type representations is native for this object’s implementation.
These methods can be used for manual control of memory deallocation.
- (buffer-refcnt buffer) → t / integer / nil
Returns t for buffers fully controlled by GC, a positive integer for reference-counted ones, and nil if the object’s memory has been released.
- (ref-buffer buffer) → buffer
If the buffer is reference-counted, increases the reference count. Returns the buffer.
- (deref-buffer buffer) → new refcnt
Dereferences the buffer if applicable. Returns the new value of buffer-refcnt. The buffer parameter may be nil.
Note: accessing a deallocated buffer results in a safe runtime exception.
- (with-deref-buffer (var expr) code…)
- (with-deref-buffers ((var expr) (var expr) …) code…)
Ensures that deref-buffer is applied to every variable once control flow leaves the code block.
- (buffer-as-array buffer &key no-copy) → array, shared?
If the buffer is actually backed by a lisp array, returns (values arr t). Otherwise, creates a compatible lisp array, copies the contents there unless no-copy is specified, and returns (values arr nil).
These methods replicate standard library functions for arrays:
- (buffer-element-type buffer) → lisp type
- (buffer-foreign-type buffer) → CFFI type
- (buffer-rank buffer) → integer
- (buffer-dimensions buffer) → integer list
- (buffer-dimension buffer axis) → integer
- (buffer-size buffer) → total size
- (buffer-row-major-index buffer &rest indexes) → integer
These methods can be used in place of aref or row-major-aref to read or write individual buffer elements:
- (bref buffer &rest indexes)
- (row-major-bref buffer index)
Note that their performance characteristics are not guaranteed to be good enough to make them useful outside of debugging code or REPL. Production code should use bulk copy functions.
- (buffer-fill buffer value &key start end)
Fills the buffer with the specified value, like the standard fill function does.
- (copy-buffer-data src-buffer src-index dest-buffer dest-index count) → count copied
Copies count consecutive elements from source to destination starting at specified indexes.
The buffers must have the same element type. If t is passed as count, the maximum possible valid value is used.
The result is undefined if the source and destination areas overlap.
- (copy-full-buffer source dest)
Copies all contents of source to dest.
The buffers must have the same element type and size. Does nothing if (eq source dest).
- (write-buffer buffer stream)
Writes the buffer contents to the binary stream.
- (read-buffer buffer stream)
Restores buffer contents from the binary stream.
Like lisp arrays, a buffer can be displaced to another buffer. All displaced views of the same underlying storage area share the reference count.
- (buffer-displacement buffer) → buffer, offset
Retrieves the displacement information for the buffer like array-displacement.
- (buffer-displace buffer &key offset byte-offset element-type foreign-type size dimensions) → displaced buffer
Creates a new displaced view based on the buffer.
The following attributes can be used to define a displaced view (each of the pairs is mutually exclusive):
- offset – initial offset in original buffer elements.
- byte-offset – initial offset in bytes.
- element-type – new lisp element type.
- foreign-type – new foreign element type.
- size – a one-dimensional size value, or t to fill all remaining space.
- dimensions – new list of dimensions.
Specific implementations of the buffer interface may support additional attributes, or impose restrictions on the argument values. For instance, standard lisp arrays don’t support changing the element type or using byte offsets that aren’t divisible by the element size.
Apart from an implementation of buffer methods for standard lisp arrays, the library includes the following buffer types:
This allocates a reference-counted buffer based on CFFI foreign memory:
- (make-foreign-array dimensions &key element-type foreign-type initial-element)
The element type defaults to single-float (:float).
If the buffer object is garbage-collected, the foreign memory is automatically released.
This allocates a reference-counted buffer based on CUDA linear memory:
- (make-cuda-array dimensions &key element-type foreign-type initial-element pitch-elt-size pitch-level)
The pitch-elt-size argument must be 4, 8 or 16. If it is specified, a pitched block is allocated. The pitch-level parameter specifies how many innermost dimensions are placed within the pitch row; it defaults to 1.
All displaced views of a pitched buffer must either fit within a pitch row, or ensure that the dimensions can be evenly split on the pitch boundary.
Memory blocks from garbage-collected buffers are collected in a queue and released once you try to allocate more memory from the same context. The memory cannot be released directly in the finalizer because of thread affinity issues.
CUDA supports allocating blocks of non-pageable host memory for asynchronous data transfer purposes. This can be exploited via the following interface:
- (make-cuda-host-array dimensions &key element-type foreign-type initial-element flags)
The function is similar to make-foreign-array, but allows one additional key argument that may contain a list of flag keywords:
-
:portable – makes the block usable from all CUDA contexts.
Without this flag the other ones handle it as a completely ordinary block of host memory.
-
:mapped – enables mapping the block into GPU memory space.
Recent NVidia devices support accessing host memory as if it was ordinary GPU memory. Exploiting this requires both creating the context with the :map-host flag and using the :mapped flag with the host allocation.
-
:write-combine – disables CPU read caching for the block.
This improves performance for device-host transfer, but makes reading the block from the CPU prohibitively expensive.
Device-host memory mapping is exposed through an additional parameter of the buffer-displace generic function:
- (buffer-displace host-buffer &key … mapping)
The following values of the mapping parameter are supported:
-
nil
A no-op value.
- :device
-
:gpu
Converts the CUDA host block buffer to a wrapper of the corresponding device-side mapping.
-
:host
Converts the device-side mapping back to a host block buffer.
Both types of conversion flags do nothing if the buffer is already of the desired type. The mapping parameter may be used together with other displacement attributes, or on its own.