OpenGVLab/InternVL3_5-241B-A28B
Image-Text-to-Text
ā¢
Updated
ā¢
2.07k
ā¢
135
Artem Darius Weber
milkomeda22
AI & ML interests
None yet
Recent Activity
liked
a model
2 days ago
OpenGVLab/InternVL3_5-241B-A28B
upvoted
a
collection
2 days ago
InternVL3.5
liked
a model
18 days ago
Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
Organizations
liked
a
model
2 days ago
upvoted
a
collection
2 days ago
liked
a
model
18 days ago
liked
a
Space
2 months ago
updated
a
Space
2 months ago
published
a
Space
2 months ago
liked
a
model
2 months ago
upvoted
a
paper
3 months ago
upvoted
an
article
3 months ago
Article
Design Patterns for Building Agentic Workflows
ā¢
9
liked
3
models
4 months ago
reacted to
codelion's
post with š„
6 months ago
Post
6195
I recently worked on a LoRA that improves tool use in LLM. Thought the approach might interest folks here.
The issue I have had when trying to use some of the local LLMs with coding agents is this:
Me: "Find all API endpoints with authentication in this codebase"
LLM: "You should look for @app .route decorators and check if they have auth middleware..."
But I often want it to search the files and show me but the LLM doesn't trigger a tool use call.
To fine-tune it for tool use I combined two data sources:
1. Magpie scenarios - 5000+ diverse tasks (bug hunting, refactoring, security audits)
2. Real execution - Ran these on actual repos (FastAPI, Django, React) to get authentic tool responses
This ensures the model learns both breadth (many scenarios) and depth (real tool behavior).
Tools We Taught:
-
-
-
-
-
-
Improvements:
- Tool calling accuracy: 12% ā 80%
- Correct parameters: 8% ā 87%
- Multi-step tasks: 3% ā 78%
- End-to-end completion: 5% ā 80%
- Tools per task: 0.2 ā 3.8
The LoRA really improves on intential tool call as an example consider the query: "Find ValueError in payment module"
The response proceeds as follows:
1. Calls
2. Gets 4 matches across 3 files
3. Calls
4. Analyzes context
5. Reports: "Found 3 ValueError instances: payment/processor.py:47 for invalid amount, payment/validator.py:23 for unsupported currency..."
Resources:
- Colab notebook https://colab.research.google.com/github/codelion/ellora/blob/main/Ellora_Recipe_3_Enhanced_Tool_Calling_and_Code_Understanding.ipynb
- Model - codelion/Llama-3.2-1B-Instruct-tool-calling-lora
- GitHub - https://github.com/codelion/ellora
The issue I have had when trying to use some of the local LLMs with coding agents is this:
Me: "Find all API endpoints with authentication in this codebase"
LLM: "You should look for @app .route decorators and check if they have auth middleware..."
But I often want it to search the files and show me but the LLM doesn't trigger a tool use call.
To fine-tune it for tool use I combined two data sources:
1. Magpie scenarios - 5000+ diverse tasks (bug hunting, refactoring, security audits)
2. Real execution - Ran these on actual repos (FastAPI, Django, React) to get authentic tool responses
This ensures the model learns both breadth (many scenarios) and depth (real tool behavior).
Tools We Taught:
-
read_file - Actually read file contents-
search_files - Regex/pattern search across codebases-
find_definition - Locate classes/functions-
analyze_imports - Dependency tracking-
list_directory - Explore structure-
run_tests - Execute test suitesImprovements:
- Tool calling accuracy: 12% ā 80%
- Correct parameters: 8% ā 87%
- Multi-step tasks: 3% ā 78%
- End-to-end completion: 5% ā 80%
- Tools per task: 0.2 ā 3.8
The LoRA really improves on intential tool call as an example consider the query: "Find ValueError in payment module"
The response proceeds as follows:
1. Calls
search_files with pattern "ValueError"2. Gets 4 matches across 3 files
3. Calls
read_file on each match4. Analyzes context
5. Reports: "Found 3 ValueError instances: payment/processor.py:47 for invalid amount, payment/validator.py:23 for unsupported currency..."
Resources:
- Colab notebook https://colab.research.google.com/github/codelion/ellora/blob/main/Ellora_Recipe_3_Enhanced_Tool_Calling_and_Code_Understanding.ipynb
- Model - codelion/Llama-3.2-1B-Instruct-tool-calling-lora
- GitHub - https://github.com/codelion/ellora
liked
5
models
6 months ago
litagin/anime-whisper
Automatic Speech Recognition
ā¢
0.8B
ā¢
Updated
ā¢
5.61k
ā¢
118
litagin/anime_speaker_embedding_ecapa_tdnn_groupnorm
Audio Classification
ā¢
Updated
ā¢
7
litagin/anime_speaker_embedding_by_va_ecapa_tdnn_groupnorm
Audio Classification
ā¢
Updated
ā¢
3
litagin/anime_speech_emotion_classification
Audio Classification
ā¢
0.2B
ā¢
Updated
ā¢
382
ā¢
5
openbmb/MiniCPM-V-4_5
Image-Text-to-Text
ā¢
Updated
ā¢
73.5k
ā¢
1.06k
reacted to
AdinaY's
post with ā¤ļø
6 months ago
Post
5069
MiniCPM-V 4.5 š New MLLM for image, multi-image & video understanding, running even on your phone, released by OpenBMB
openbmb/MiniCPM-V-4_5
āØ SOTA vision language capability
āØ 96Ć video token compression > high-FPS & long video reasoning
āØ Switchable fast vs deep thinking modes
āØ Strong OCR, document parsing, supports 30+ languages
openbmb/MiniCPM-V-4_5
āØ SOTA vision language capability
āØ 96Ć video token compression > high-FPS & long video reasoning
āØ Switchable fast vs deep thinking modes
āØ Strong OCR, document parsing, supports 30+ languages
liked
a
dataset
6 months ago