Pinned Loading

1. llm-inference-stack llm-inference-stack Public

   Fifteen LLM inference systems on one shared core: serving, TTFT/ITL benchmarking, KV cache budgeting, prefix-aware routing, quantization, speculative decoding, Triton kernels, chunked prefill, Page…

   Python

2. spark-rapids-tools spark-rapids-tools Public

   Forked from NVIDIA/cudf-spark-tools

   User tools for Spark RAPIDS

   Scala 1

3. ray ray Public

   Forked from ray-project/ray

   Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

   Python

4. lws lws Public

   Forked from kubernetes-sigs/lws

   LeaderWorkerSet: An API for deploying a group of pods as a unit of replication

   Go 1

5. llm-d-inference-scheduler llm-d-inference-scheduler Public

   Forked from llm-d/llm-d-router

   Inference scheduler for llm-d

   Go

6. data-on-eks data-on-eks Public

   Forked from awslabs/data-on-eks

   DoEKS is a tool to build, deploy and scale Data Platforms on Amazon EKS

   Shell