Models
Datasets
Spaces
Docs
Enterprise
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2309.01940

Code LMs Evaluation

Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code

Paper • 2311.07989 • Published Nov 14, 2023 • 26
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Paper • 2310.06770 • Published Oct 10, 2023 • 9
CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

Paper • 2401.03065 • Published Jan 5, 2024 • 11
Copilot Evaluation Harness: Evaluating LLM-Guided Software Programming

Paper • 2402.14261 • Published Feb 22, 2024 • 10

Creative Robot Tool Use with Large Language Models

Paper • 2310.13065 • Published Oct 19, 2023 • 9
CodeCoT and Beyond: Learning to Program and Test like a Developer

Paper • 2308.08784 • Published Aug 17, 2023 • 5
Lemur: Harmonizing Natural Language and Code for Language Agents

Paper • 2310.06830 • Published Oct 10, 2023 • 33
CodePlan: Repository-level Coding using LLMs and Planning

Paper • 2309.12499 • Published Sep 21, 2023 • 79

Dissecting In-Context Learning of Translations in GPTs

Paper • 2310.15987 • Published Oct 24, 2023 • 6
Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca

Paper • 2309.08958 • Published Sep 16, 2023 • 2
X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languages

Paper • 2305.04160 • Published May 7, 2023 • 2
Ziya-VL: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuning

Paper • 2310.08166 • Published Oct 12, 2023 • 1

Multiple choice Q&A

Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code

Paper • 2303.08033 • Published Mar 9, 2023 • 1
CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Models

Paper • 2309.01940 • Published Sep 5, 2023 • 2

KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval

Paper • 2310.15511 • Published Oct 24, 2023 • 5
HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models

Paper • 2310.14566 • Published Oct 23, 2023 • 27
SmartPlay : A Benchmark for LLMs as Intelligent Agents

Paper • 2310.01557 • Published Oct 2, 2023 • 13
FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation

Paper • 2310.03214 • Published Oct 5, 2023 • 20

Code LMs Evaluation

Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code

Paper • 2311.07989 • Published Nov 14, 2023 • 26
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Paper • 2310.06770 • Published Oct 10, 2023 • 9
CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

Paper • 2401.03065 • Published Jan 5, 2024 • 11
Copilot Evaluation Harness: Evaluating LLM-Guided Software Programming

Paper • 2402.14261 • Published Feb 22, 2024 • 10

Multiple choice Q&A

Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code

Paper • 2303.08033 • Published Mar 9, 2023 • 1
CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Models

Paper • 2309.01940 • Published Sep 5, 2023 • 2

Creative Robot Tool Use with Large Language Models

Paper • 2310.13065 • Published Oct 19, 2023 • 9
CodeCoT and Beyond: Learning to Program and Test like a Developer

Paper • 2308.08784 • Published Aug 17, 2023 • 5
Lemur: Harmonizing Natural Language and Code for Language Agents

Paper • 2310.06830 • Published Oct 10, 2023 • 33
CodePlan: Repository-level Coding using LLMs and Planning

Paper • 2309.12499 • Published Sep 21, 2023 • 79

KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval

Paper • 2310.15511 • Published Oct 24, 2023 • 5
HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models

Paper • 2310.14566 • Published Oct 23, 2023 • 27
SmartPlay : A Benchmark for LLMs as Intelligent Agents

Paper • 2310.01557 • Published Oct 2, 2023 • 13
FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation

Paper • 2310.03214 • Published Oct 5, 2023 • 20

Dissecting In-Context Learning of Translations in GPTs

Paper • 2310.15987 • Published Oct 24, 2023 • 6
Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca

Paper • 2309.08958 • Published Sep 16, 2023 • 2
X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languages

Paper • 2305.04160 • Published May 7, 2023 • 2
Ziya-VL: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuning

Paper • 2310.08166 • Published Oct 12, 2023 • 1

Company

TOS Privacy About Careers

Website

Models Datasets Spaces Pricing Docs