Datasets from the paper "Towards Evaluation of Implicit Software World Models in Coding LLMs"
AI & ML interests
None defined yet.
Recent Activity
View all activity
Papers
On Pretraining for Project-Level Code Completion
Diff-XYZ: A Benchmark for Evaluating Diff Understanding
Organization Card
At JetBrains we are building AI-assisted developer tools to free developers from repetitive tasks, help them stay in the flow and produce high quality software. We are working on various code processing tasks such as code completion, code generation, code summarization, diff processing, code editing and more.
Our specific areas of interest include (but are not limited to!) the following:
- High-quality ML models for every programming language
- Training materials and techniques for low-resource languages
- Improving models’ performance with feedback from code analysis tools
- Adapting models to the software project at hands
- Long-context models and their evaluation
- Efficient in-project retrieval
- Agents that can perform software engineering tasks
- Issue-solving agents
- Debugging agents
Learn more about us: JetBrains Research | JetBrains AI Assistant | JetBrains AI
All the checkpoints from Table 3 of the paper “On Pretraining for Project-Level Code Completion.”
-
On Pretraining for Project-Level Code Completion
Paper • 2510.13697 • Published • 7 -
JetBrains-Research/OpenCoder-1.5B-File-Level-4K-without-Theta-Scaling
Text Generation • 2B • Updated • 7 -
JetBrains-Research/OpenCoder-1.5B-File-Level-4K-with-Theta-Scaling
Text Generation • 2B • Updated • 5 -
JetBrains-Research/OpenCoder-1.5B-Path-Distance-Py
Text Generation • 2B • Updated • 7
Datasets from the paper "Towards Evaluation of Implicit Software World Models in Coding LLMs"
All the checkpoints from Table 3 of the paper “On Pretraining for Project-Level Code Completion.”
-
On Pretraining for Project-Level Code Completion
Paper • 2510.13697 • Published • 7 -
JetBrains-Research/OpenCoder-1.5B-File-Level-4K-without-Theta-Scaling
Text Generation • 2B • Updated • 7 -
JetBrains-Research/OpenCoder-1.5B-File-Level-4K-with-Theta-Scaling
Text Generation • 2B • Updated • 5 -
JetBrains-Research/OpenCoder-1.5B-Path-Distance-Py
Text Generation • 2B • Updated • 7
spaces 7
Paused
ML4SE Benchmark Viewer
📊
Explore ML4SE benchmark problems with filters and search
Running
Agents
42
Long Code Arena
🏟
View model performance leaderboards for various tasks
Sleeping
Agents
Routing Money Calculation
🌍
Rough estimate of routing cost
Runtime error
Agents
Commit Message Editing
✍
Sleeping
Agents
commit-labeling
🚀
models 53
JetBrains-Research/ltmia-code-mia-classifier
Updated
JetBrains-Research/doc2lora-niah
Updated • 4
JetBrains-Research/learned-transfer-attack
Other • Updated
JetBrains-Research/Qwen3-30B-A3B-am
31B • Updated • 2
JetBrains-Research/rocq-language-theorem-embeddings
0.1B • Updated • 6
JetBrains-Research/OpenCoder-1.5B-Masked-Leak
Text Generation • 2B • Updated • 5 • 1
JetBrains-Research/OpenCoder-1.5B-Leak-Irrelevant
Text Generation • 2B • Updated • 5
JetBrains-Research/OpenCoder-1.5B-Leak-Reversed
Text Generation • 2B • Updated • 3
JetBrains-Research/OpenCoder-1.5B-Leak
Text Generation • 2B • Updated • 3 • 1
JetBrains-Research/OpenCoder-1.5B-Duplication
Text Generation • 2B • Updated • 5
datasets 42
JetBrains-Research/cwm-benchmarks-dl4c-traces
Viewer • Updated • 435 • 179
JetBrains-Research/cwm-benchmarks-dl4c-benchmark
Viewer • Updated • 435 • 153
JetBrains-Research/cwm-benchmarks-dl4c-environments
Viewer • Updated • 217 • 150
JetBrains-Research/nes-mixed-v9-memorization
Viewer • Updated • 1.87M • 1.48k
JetBrains-Research/django_method_gen
Viewer • Updated • 1.46k • 26
JetBrains-Research/agent-trajectories-swe-bench-test-minus-verified
Viewer • Updated • 1.79k • 120
JetBrains-Research/agent-trajectories-swesmith-random-subset
Viewer • Updated • 1.47k • 83
JetBrains-Research/ltmia-data
Updated • 32
JetBrains-Research/REval
Viewer • Updated • 1.7k • 25 • 1
JetBrains-Research/PIPer-eval
Preview • Updated • 534