publications

2026

1. MLSys

   FaaScale: Unlocking Fast LLM Scaling for Serverless Inference

   Minchen Yu, Rui Yang, Chaobo Jia, Zhaoyuan Su, Sheng Yao, Tingfeng Lan, Yuchen Yang, Yue Cheng, Wei Wang, Ao Wang, and Ruichuan Chen

   in the Proceedings of the 9th Annual Conference on Machine Learning and Systems, May 2026

   arXiv

2025

1. SoCC

   ZipBatch: Multi-Tenant GPU Batching with Dual-Resource Regulation

   Haoxuan Yu, Sheng Yao, and Wei Wang

   in the Proceedings of the 2025 ACM Symposium on Cloud Computing, Nov 2025

   Abs DOI PDF

   GPU multiplexing is a widely-adopted strategy in GPU clusters for improving overall throughput and lowering the total cost of ownership. To mitigate inter-task interference in compute power and memory bandwidth on multiplexed GPUs, existing techniques divide a GPU into instances with limited predefined rigid configurations. Low utilization arises from the mismatch between heterogeneous burstiness and immutable resource configurations: 1) bursty inference traffic forces the scheduler to launch underfilled batches that cannot saturate the instance; 2) bursty kernel resource utilization leads to bubbles in compute power and memory bandwidth. This paper presents ZipBatch, a multi-tenant model serving system with elastic fine-grained GPU resource provisioning. First, ZipBatch launches dynamic batches with right-sized resource reservations to reduce macroscopic model-level over-provisioning while decreasing queuing time. Second, by observing that kernels exhibit imbalanced and inconsistent utilization of resources, we develop a prediction-based dual-dimensional regulator that allocates right-sized compute power and memory bandwidth in real time to suppress microscopic resource underutilization. Compared with state-of-the-art solutions, our experimental results show that ZipBatch achieves up to 60.84% reduction in average latency while increasing total throughput by 71.27%.

2024

1. SIGMOD

   WeBridge: Synthesizing Stored Procedures for Large-Scale Real-World Web Applications

   Gansen Hu, Zhaoguo Wang, Chuzhe Tang, Jiahuan Shen, Zhiyuan Dong, Sheng Yao, and Haibo Chen

   in the Proceedings of the 2024 ACM SIGMOD International Conference on Management of Data, Jun 2024

   Abs DOI PDF

   Modern web applications use databases to store their data. When processing user requests, these applications retrieve and store data in the database server, which incurs network round trips. These round trips significantly increase the application’s latency. Previous approaches have attempted to reduce these round trips by prefetching query results or batching database accesses. However, neither method can efficiently reduce the latency when some queries depend on previous queries’ results. In real-world applications, nearly 50% of the queries depend on the result of other queries.This paper presents WeBridge, the first system capable of synthesizing stored procedures for large-scale real-world web applications. First, WeBridge employs concolic execution technique to analyze the applications and generate stored procedures for hot program paths. Then, it seamlessly integrates the stored procedures into the application by extending the database access library. Finally, it improves the efficiency of the stored procedures with speculative execution. Evaluation using real-world web applications and workloads show that WeBridge achieves up to 79.8% median latency reduction and up to 2\texttimes peak throughput.