Artificial Intelligence

Fourier Position Embedding: Enhancing Attentions Periodic Extension for Length Generalization
Extending the context length of Language Models (LMs) by improving Rotary Position Embedding (RoPE) has become a trend. While existing …
PDF Cite Code
Fourier Position Embedding: Enhancing Attentions Periodic Extension for Length Generalization
Free Process Rewards without Process Labels
Different from its counterpart outcome reward models (ORMs), which evaluate the entire responses, a process reward model (PRM) scores a …
PDF Cite Code
Free Process Rewards without Process Labels
How to Synthesize Text Data without Model Collapse?
Model collapse in synthetic data indicates that iterative training on self-generated data leads to a gradual decline in performance. …
PDF Cite
How to Synthesize Text Data without Model Collapse?
MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding
We introduce MedXpertQA, a highly challenging and comprehensive benchmark to evaluate expert-level medical knowledge and advanced …
PDF Cite Code
MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding
GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning
Recent advancements in Large Language Models (LLMs) have shown that it is promising to utilize Process Reward Models (PRMs) as …
PDF Cite Code
GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning
A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond.
Recent Large Reasoning Models (LRMs), such as DeepSeek-R1 and OpenAI o1, have demonstrated strong performance gains by scaling up the …
PDF Cite
A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond.
Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models
State Space Models (SSMs) have emerged as a promising alternative to the popular transformer-based models and have been increasingly …
PDF Cite
Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models
Less is More: Efficient Model Merging with Binary Task Switch
As an effective approach to equip models with multi-task capabilities without additional training, model merging has garnered …
PDF Cite Code
Less is More: Efficient Model Merging with Binary Task Switch
Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
Test-Time Scaling (TTS) is an important method for improving the performance of Large Language Models (LLMs) by using additional …
PDF Cite Code
Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
Process reinforcement through implicit rewards.
Dense process rewards have proven a more effective alternative to the sparse outcome-level rewards in the inference-time scaling of …
PDF Cite Code
Process reinforcement through implicit rewards.