Agentic Search for Context Engineering

This post is an edited long-form version of the workshop titled "Agentic Search for Context Engineering" I gave at AI Engineer Europe 2026 on April 8, 2026 in London. The slides, code, and diagrams are available in the workshop repository. Full recording on YouTube: https://www.youtube.com/watch?v=ynJyIKwjonM

If you've built an agent before, you know that context engineering is a crucial part of making an agent return meaningful responses.

Context engineering is the process of deciding what from all possible context sources actually goes into the agent's context window so the LLM can generate the best response. This is also referred to as the process of "context curation" — the arrow from the possible context sources to the context window.

I think we don't give this context curation arrow enough credit because it does almost all of the heavy lifting. What hides behind it is a set of search tools the agent can decide to use. That's why my personal hot take is that:

Context engineering is about 80% agentic search.

History

How search in the AI stack has evolved from Retrieval-Augmented Generation (RAG), to agentic search/agentic RAG, and now to context engineering in the last three years.

Retrieval-Augmented Generation

When we first started building with LLMs in 2024, we started implementing RAG systems with fixed retrieval pipelines:

1. The user message is used - more or less verbatim - as a search query (often vector search).
2. Chunks are pulled from a database once.
3. Retrieved chunks are combined with the user message in the prompt and fed to the LLM.

This design is straightforward and still works for narrow Q&A. But it breaks in predictable ways because you retrieve exactly once:

• You retrieve once even when the model does not need external context, and irrelevant chunks can confuse it.
• You retrieve once and have no option to correct the query. What if the returned results don't contain the relevant information and you'd need to run a second search?
• You retrieve once even when the question needs multi-hop retrieval. The first batch of chunks might only tell you what to search for next, but the pipeline never runs a second pass.

Agentic RAG

To overcome these limitations, we replaced the fixed pipeline with a search tool and called it "agentic RAG", "agentic retrieval", or "agentic search". In this scenario, the agent decides whether to call a search tool, whether results are relevant, whether to retrieve more, and whether to rewrite the search query.

In many setups this is straightforward because you often have only one or a limited amount of context sources and one retrieval tool.

Context engineering

In context engineering we now have many different context sources:

• Local files (your repo, scratch pads with plan.md or todo.md files, or Agent Skills) on disk,
• Databases (storing large-scale enterprise data),
• Web,
• Long-term memory (whether memory lives in files or a database is still debated).

Depending on the context source, we often have a native search tool:

• file search for local files,
• skill loading for Agent Skills,
• dedicated database tools (semantic search or query execution) for databases,
• web search for the web, and
• memory tools for long-term memory.

If that's not overwhelming enough, we now also have a tool that lets the agent run terminal commands. This tool has many different names: LangChain calls it the "shell tool", Anthropic the "bash tool", OpenClaw the "exec tool". The post refers to it as the "shell tool".

The shell tool is a versatile tool because it can interact with most context sources: It can run against local files (ls, grep), databases (CLIs, scripts, curl to HTTPS APIs), and the web (curl).

There's been a lot of discussion whether a shell tool is all an agent needs. The author's related blog: "The shell tool is not a silver bullet for context engineering". The TLDR: the practical question is which search tools belong in your stack, not shell tool versus all others.

Doing good search is difficult. That is why we have so many different search techniques and why you curate a stack for your latency and quality requirements.

Code walkthrough

Uses LangChain for orchestration. Data: the AI Engineer Europe 2026 schedule. First two demos use a local Elasticsearch cluster; the third reads the same data from disk.

Vanilla agentic search

Minimal setup: one semantic search tool over the index conference_schedule (one document per session). LLM is gpt-5.4-nano through LiteLLM. A minimal system prompt tells the agent it's a search agent, when to retrieve, and how the index is shaped. Embedding model is jina-embeddings-v5-text-small, connecting to an ElasticsearchStore.

The search tool semantic_search_conference_sessions(query) runs a semantic search and returns the top 3 results, wired up with LangChain's @tool decorator (function name → tool name, docstring → description).

@tool()
def semantic_search_conference_sessions(query: str) -> str:
    """Runs a semantic search query to find conference sessions by concept or topic."""
    docs = vector_store.similarity_search(query, k=3)
    return "\n\n".join(
        f"**{doc.metadata['type']} by {doc.metadata['speakers']}**: "
        f"{doc.metadata['title']}\nDescription:{doc.page_content}"
        for doc in docs
    )

For a semantic query like "Which sessions discuss regulatory constraints in AI systems?" the agent finds the right talk. This is where most agentic search demos stop.

But to break a semantic-search-only demo, ask something where a keyword-based search is more suitable. For "Which sessions should I visit to learn more about GEPA?" the semantic search returns unrelated sessions (it confuses GEPA with "Gemma") and misses the correct session.

Agentic search with general-purpose database search tool

Replace the narrow semantic tool with a more general-purpose tool execute_esql_query that lets the agent write full ES|QL queries against Elasticsearch. Switch from gpt-5.4-nano to gpt-5.4-mini because query generation is harder than passing a topic. Add a try/except that returns meaningful error info so the agent can self-correct.

@tool()
def execute_esql_query(esql_query: str) -> str:
    """Execute an ES|QL query against the conference_schedule index in Elasticsearch."""
    try:
        response = es_client.esql.query(query=esql_query, format="csv")
        return response.body
    except Exception as e:
        return f"Error executing ES|QL query: {e}"

First attempt fails: the agent writes WHERE text LIKE '%GEPA%' using SQL-style % wildcards, but ES|QL uses * for wildcards, so it returns zero results. (Also: decide in your product whether zero rows means "nothing found" or "retry with different parameters.")

Fix via Agent Skills: define a minimal elasticsearch-esql skill (name, description, content with ES|QL syntax rules), then use the LangChain skill-loading pattern (load_skill tool + SkillMiddleware that injects skill descriptions and registers load_skill). Tell the agent — via both the tool description and the system prompt — to always call the ES|QL skill before executing a query, and to regenerate on error.

After loading the skill, the agent writes valid ES|QL (LIKE "*GEPA*") and finds the right session (Samuel Colvin's "Playground in Prod"). Because the tool is general-purpose, the agent can also answer analytical questions like "How many sessions are on April 8?" using STATS count = COUNT() — useful because LLMs are notoriously bad at counting. (Result: 27.)

Trade-off: execute_esql_query is more powerful than the narrow semantic_search tool for ambiguous/complex queries, but skill loading adds cost and latency and requires a more powerful LLM.

Agentic search with shell tool

"Bash + Filesystem is all you need." Move the data from Elasticsearch to local files under ../data/session_data/ (one .txt per session), and let the agent search over them with LangChain's ShellTool.

shell_tool = ShellTool()  # no safeguards by default; sandbox in production

For "Are there any sessions about GEPA?" the agent explores the filesystem then greps for "GEPA" and finds the right session.

But for the semantic query "Which sessions discuss handling regulatory constraints?", you see how agents can cheat at semantic search with grep: the agent starts by looking for "regulat", then strings together related synonyms (compliance, constraints, GDPR, governance) until something hits. This works almost unreasonably well, but is it the most effective? The agent would have to chain together all animals if asked to find "movies with animal superheroes."

That's why we're seeing many semantic search alternatives to grep: LlamaIndex's semtools, LightOn's colgrep, and Jina AI's jina-grep-cli.

Giving the agent jina-grep-cli: install the CLI, then describe it and its usage in the system prompt (grep-compatible flags -r/-R/-l, semantic flags --threshold for cosine similarity default 0.5, --top-k default 10). The prompt also gives a grep vs jina-grep decision rule:

• Exact substring, known filename, or simple listing → grep / find / cat.
• Natural-language or fuzzy "what talks mention X?" over many .txt files locally → jina-grep (only run one at a time; don't chain multiple jina-grep commands).

With this loaded, the agent finds the right session for the regulatory-constraints query more efficiently, returning cosine-similarity-scored matches.