Zhang Zhixia

Peking University · Incoming Ph.D. Student · Fall 2027

Zhang Zhixia 张之夏

About Me

I am an undergraduate student in Computer Science and Technology at Beihang University. In Fall 2027, I will join the Discovery Intelligence Group (DIG) at Peking University as a direct-entry Ph.D. student, advised by Prof. Ling Yang.

My research interests lie in reinforcement learning, large language models, and AI agents. I study how agents learn from interaction and experience to improve reasoning and decision-making. My recent work explores learning signals, credit assignment, and agent collaboration for complex reasoning tasks.

My long-term goal is to build agent systems that autonomously evolve and reliably carry out long-horizon tasks. I am interested in how agents reuse past experience, adapt to new challenges, and turn these capabilities into practical solutions for real-world problems.

Research interests

Reinforcement learningLarge language modelsAutonomous agentsLong-horizon reasoning

Selected Papers

* Equal contribution, unless otherwise noted.

HACRL framework for collaborative reinforcement learning with shared rollouts from heterogeneous agents.
Accepted
Accepted · NeurIPS 2026HACRL

Heterogeneous Agent Collaborative Reinforcement Learning

Zhang Zhixia*, Zixuan Huang*, Gongxun Li, Huaiyang Wang, Chengyi Yuan, Xin Xia, Deqing Wang, Fuzhen Zhuang, Shuai Ma, Ning Ding, Yaodong Yang, Jianxin Li, Yikun Ban

Enables heterogeneous agents to learn from shared verified rollouts while remaining independent at inference time.

arXiv Code
MS-TTRL framework with outcome self-verification, a three-state cache, and confidence-gated reward assignment.
ManuscriptMS-TTRL

MS-TTRL: Breaking the Frequency Conflict in Multiple-Solution Test-Time Reinforcement Learning

Zhang Zhixia, Bingqian Xu, Huaiyang Wang, Yikun Ban, Dianbo Liu

Decouples self-verified correctness from outcome frequency, using a three-state cache and confidence-gated scaling to preserve diverse solutions during test-time reinforcement learning.

Five levels toward AGI connect exploration spaces with information gain, value improvement, and epistemic reachability.
Survey · PreprintAgent Exploration

Agent Exploration Toward Artificial General Intelligence: A Survey

Yikun Ban, Fengkai Yang*, …, Zhang Zhixia, et al.All 36 authorsFewer authors

Yikun Ban, Fengkai Yang*, Fangzheng Chen*, Yibo Wang*, Zhijun Chen*, Zhongyi Li*, Gongxun Li, Huaiyang Wang, Xiaodong Lu, Zixuan Huang, Xiaoyuan Zhang, Zehao Chen, Yaocheng Yang, Pengcheng Wei, Wan Tian, Zherui Chen, Zhang Zhixia, Hongyan Xie, Lingyu Lv, Xiaohan Wang, Jiajun Chai, Guojun Yin, Wei Lin, Tianxiang Ai, Zhongxiang Dai, Zhiyong Wang, Ruijie Wang, Haoyi Zhou, Chaochao Lu, Wanxiang Che, Fuzhen Zhuang, Ning Ding, Qianqian Xu, Deqing Wang, Yaodong Yang, Jianxin Li

Presents exploration as a mechanism for progressing from reactive models to reasoning agents, prospectors, and collaborative ecosystems.

Pre-print GitHub
SentencePO framework for sentence-level advantage estimation, credit assignment, and trust-region optimization.
ManuscriptSentencePO

SentencePO: Sentence-Level Trust Region and Credit Assignment for Reinforcement Learning with Verifiable Rewards

Ziyuan Wang, Fengkai Yang, Yumo Jiang, Zhang Zhixia, Yikun Ban, Xiaohan Wang, Jiajun Chai, Guojun Yin, Wei Lin, Deqing Wang, Fuzhen Zhuang

Uses sentence-level trust regions and credit assignment to improve the stability and precision of reinforcement learning for long-form reasoning.

SAGE identifies efficient reasoning paths and trains models to improve accuracy while using fewer tokens.
Accepted
Accepted · ICML 2026SAGE-RL

Does Your Reasoning Model Implicitly Know When to Stop Thinking?

Zixuan Huang, Xin Xia, Yuxi Ren, Jianbin Zheng, Xuanda Wang, Zhang Zhixia*, Hongyan Xie, Songshi Liang, Zehao Chen, Xuefeng Xiao, Fuzhen Zhuang, Jianxin Li, Deqing Wang, Yikun Ban

* Second student author

Uses self-aware sampling and reinforcement learning to help reasoning models find concise, effective chains of thought.

arXiv
PIPO closes the reinforcement learning loop with policy improvement verification and adaptive updates.
PreprintPIRL / PIPO

Policy Improvement Reinforcement Learning

Huaiyang Wang*, Xiaojie Li*, Xiaohan Wang, Zhang Zhixia, Xiaodong Lu, Zixuan Huang, Jiajun Chai, Guojun Yin, Deqing Wang, Haoyi Zhou, Yaodong Yang, Jianxin Li, Yikun Ban

Closes the policy optimization loop by verifying whether each update improves performance and using that feedback to regulate learning.

arXiv Code
Bayesian trading framework linking an adversarial synthetic market to a robust trading agent.
Accepted
Accepted · KDD 2026Bayesian Robust Trading

Bayesian Robust Financial Trading with Adversarial Synthetic Market Data

Haochong Xia*, Simin Li*, Ruixiao Xu, Zhang Zhixia, Hongxiang Wang, Zhiqian Liu, Teng Yao Long, Molei Qin, Chuqiao Zong, Bo An

Combines macro-conditioned synthetic markets with Bayesian adversarial learning to improve trading robustness under changing market regimes.

arXiv Code
Transferable adversarial policy framework combining tactic acquisition with scene decomposition.
ManuscriptAdversarial Policy Transfer

Adversarial Policy Transfer in Mixed Cooperative-Competitive Games

Ruixiao Xu, Zhiqian Liu, Zhang Zhixia, Simin Li, Yaodong Yang, Xianglong Liu

Transfers adversarial tactics across cooperative-competitive multi-agent games to reveal shared vulnerabilities and improve robustness.