Repository navigation
[RFC]: DLPack based exchange of dtype and Device #972
Description
Background
As of now we have standard interface for exchanging tensors through DLPack. One minor but still useful thing is to also provide a mechanism for solutions. Motivating example
import numpy
import torch
# results in error
torch.empty((1, 2), dtype=numpy.float16)
# results in error
numpy.empty((1, 2), dtype=torch.float16)It could be useful for the package.dtype and package.Device to be able to provide an interface for exchange such information. Given we already have DLDataType and DLDevice in dlpack, it might be helpful to bringup interfaces to transparently expose them. Such information can also be helpful for bindings to implement generic support for these types
Proposal
# defined in dtype
class dtype:
def __dlpack_data_type__():
"""Return a tuple (code, bits, lanes) in DLDataType format
"""
# defined in Device
class Device:
def __dlpack_device__():
"""Return a tuple of (device_type, device_id) """Then if desirable, the interface could implement something like
def empty(shape, dtype):
if hasattr(dtype, "___dlpack_data_type__"):
convertThere is also possibility of trying to run general value conversion. I do not think we have to update from_dlpack to also support these values, but there is an option to do so
def from_dlpack(src):
if hasattr(src, "__dlpack__"):
# this is an array
if hasattr(src, "__dlpack_data_type__"):
# this is a dtype
if hasattr(src, "__dlpack_device__"):
# this is a device
Activity
[-][RFC] DLPack based exchange of dtype and Device[/-][+][RFC]: DLPack based exchange of dtype and Device[/+]rgommers commentedon Jul 21, 2026
Thanks for the proposal @tqchen, and sorry that no one responded yet (bandwidth problem). This is actually a good idea I think; it has come up multiple times now. We deliberately avoided making dtypes and devices interchangeable at the start of the array API standard design, because (a) we didn't want to, and would not have been able to, prescribe specific implementations (e.g., class-based, anything beyond a name really for dtypes), and (b) the use cases were pretty thin. You can actually do it today, with some effort, since the underlying DLPack metadata does contain the relevant information. So at this stage we have a better understanding of needs, and can try it out for existing adopters to ensure it is actually fit for purpose.
Device is more important than dtype probably (and more challenging), since there's much less about devices that is standardized.
I guess an obvious question for
Deviceis whether(device_type, device_id)is enough or whether it should be able to contain stream information. Also, SYCL is going to bring up the two-level device thing they have again.tqchen commentedon Jul 21, 2026
agree that device context is going to be much more complicated, so original intention was to keep it simple as
(device_type, device_id). generally for such exchange to work, some form of global consensus is helpful, and atm it is limited to cuda/rocm and other device that may expose a cuda like apirgommers commentedon Jul 21, 2026
Makes sense, CUDA/ROCm covers the vast majority of usage, and at least for the cases I'd like to use it for (which is a small sample size compared to yours I suspect), what one needs to know is device equality only (stream handling is already done under the hood of
__dlpack__and I don't see a reason to lift it up yet).