AI Agents Research-to-Code: Build Skills Without Fine-Tuning

Why an agent skill can change behavior without changing model weights

An agent’s behavior depends on more than its underlying model. The software harness around it can provide instructions, tools, workflows, and rules for handling results. A skill uses that surrounding system to guide the agent through a particular kind of task, without changing the model’s weights.

That distinction matters. If a paper describes a method for analyzing documents, for example, the skill might tell the agent when to use the method, how to extract relevant evidence, and what format to return. It could also provide a document-parsing tool or a small program for checking its calculations.

Reporting on coding agents has highlighted the role of this harness: a coding agent can improve its capabilities by changing the software that turns its reasoning into actions. That is not the same as proving a general paper-to-skill process, but it shows why the implementation layer matters. The model proposes or selects actions; the harness determines what actions are available and how they are carried out.

Extract the paper’s method before writing code

Start by turning the paper into a short, structured description. Don’t begin by pasting the whole paper into an agent and asking it to “implement the method.” That leaves the task underspecified and makes it difficult to tell whether the result matches the research.

Record these elements:

• Task: What problem does the method solve, and what counts as a successful result?
• Inputs: What information, data, or tools does it require?
• Outputs: What should it produce, and in what format?
• Procedure: What steps does the method follow? Note decision points, not just the broad phases.
• Assumptions: What must be true about the data, environment, or user request?
• Evaluation: Which examples, metrics, or benchmarks did the paper use?

Separate the authors’ stated method from your interpretation. If a paper says to “select the most relevant evidence” but does not define relevance, mark that as an open question. Don’t quietly turn a missing detail into a fact about the original method.

A hypothetical example: suppose a paper describes a process for comparing product policies. Its task might be to identify changes between two documents; inputs are the documents and a comparison question; output is a list of changes with supporting passages. The method might require matching sections, checking dates, and distinguishing direct contradictions from additions. That outline is already more useful for implementation than a general instruction to “compare these policies.”

Write the method as a skill specification

Next, turn the extracted method into a specification an agent and its harness can follow. Keep it concrete enough to test.

Define:

A useful skill specification also makes boundaries explicit. If the paper’s method was evaluated on short English documents, don’t imply that it is validated for scanned contracts in several languages. Record that gap in the skill’s instructions and tests.

Implement the skill in the agent’s harness

Build the smallest version that can carry out the specified procedure. Depending on the agent framework, the skill might consist of instructions, a tool, a sequence of tool calls, or a combination of these. Keep the model-facing guidance separate from deterministic code where possible: instructions can explain choices, while code can handle repeatable operations such as parsing files or validating output fields.

Start with a minimal tool interface. For the policy-comparison example, that might be a function that accepts two documents and returns their text with section labels. The agent can then apply the paper’s comparison procedure and return a structured list of differences. Give the tool clear input limits and error messages rather than letting missing data produce plausible-looking results.

This is the paper-to-code workflow in miniature: convert a method into a specification, map the steps to agent actions, and make the result observable. Avoid building a large system before you know which parts of the method are actually needed.

Test the skill against examples, then iterate

Test the implementation on examples from the paper when they are available. Compare the agent’s outputs with the reported results, using the paper’s metrics or criteria where possible. Also include cases that expose likely errors: incomplete inputs, ambiguous instructions, and examples outside the method’s stated assumptions.

Treat reproduction as verification, not proof. Matching a paper’s examples shows that the implementation can handle those cases; it does not establish that the method works broadly or that the agent will behave reliably in a live setting. The available reporting on agent-development systems does not establish benchmark results for a general paper-to-agent-skill workflow.

When a test fails, identify which layer caused the failure. Did the instructions omit a decision rule? Did the tool return incomplete information? Did the model ignore a required step? Did the evaluation expect something the paper never specified? Fix the relevant layer - skill instructions, code, or tool interface - and rerun the test. Adding more prompt text is not automatically the right answer.

This iterative approach resembles reported agent-development systems in which experiments produce results that inform what to try next. It is a useful development pattern, not evidence that an agent can automatically turn any paper into a dependable skill.

Know what paper-to-skill workflows can and cannot prove

A paper may omit details needed to reproduce its results, or use data and evaluation conditions that are hard to match. An agent may also produce a correct-looking answer for the wrong reason. Use checks that expose its evidence and intermediate outputs, and test it on realistic cases before relying on it.

Keep skill acquisition separate from open-ended agent exploration. In a reported experiment, agents left without a specific task in a simulated environment began inventing challenges. That illustrates a different research direction; it does not show that agents can extract reliable skills from research papers on their own.

External skills are not a guarantee that fine-tuning is unnecessary. If the desired capability depends on learned patterns that the model consistently lacks, or if the skill cannot reliably steer the model, another approach may be needed. The practical advantage is that a skill prototype is inspectable and easy to revise before you invest in more complex changes.

Choose one paper with a clear, testable method. Write its skill specification, implement only the tools it needs, and compare its outputs with the paper’s examples before expanding it.

Step-by-Step Guide

1. 1

   Extract the paper’s method

   Record the task, inputs, outputs, procedure, assumptions, and evaluation criteria. Mark unclear or missing details instead of silently filling them in.

2. 2

   Write a testable skill specification

   Define when the skill should be used, its ordered steps, required tools, output format, evidence requirements, and conditions for stopping or reporting uncertainty.

3. 3

   Implement the smallest useful version

   Map the specified steps to agent instructions and tools. Use deterministic code for repeatable tasks such as parsing documents or validating output fields.

4. 4

   Test against examples and edge cases

   Compare outputs with the paper’s examples and evaluation criteria where possible. Add tests for incomplete inputs, ambiguity, and cases outside the method’s assumptions.

5. 5

   Diagnose failures and iterate

   Identify whether an error came from instructions, tools, model behavior, or evaluation assumptions. Update the relevant layer and rerun the tests.

Key Statistics

• The article’s paper-extraction checklist contains 6 elements: task, inputs, outputs, procedure, assumptions, and evaluation.This is a count of the workflow components presented in the article, not an empirical research finding.
• The article’s skill specification covers 5 areas: when to use the skill, steps, tools, expected output, and failure conditions.This is a count of the recommended specification areas in the article. The article reports no benchmark statistic for a general paper-to-skill workflow.

Frequently Asked Questions