Elasticsearch Semantic Search and AI Agent Development

Keynote
  • Liu Xiaoguo Elastic Chief Evangelist, China Community

    Currently the Chief Evangelist for the Elastic China community. Holds a Master's degree from the National University of Singapore and Bachelor's and Master's degrees from Northwestern Polytechnical University. Previously worked at companies including Singapore Technologies, Compaq, General Motors, Ericsson, Nokia, the non-profit organization Linaro (Linux for ARM), Ubuntu, and Vantiq. Has experience in computer design, automotive electronics, computer operating systems, telecommunications, and cloud real-time event processing. Has nearly 20 years of experience in community work, starting from Ericsson, then Nokia, Ubuntu, and now Elastic. Enjoys sharing the knowledge he has learned. Believes that helping others is helping oneself. Hopes to share and learn with everyone. Welcome to visit the official Elastic Chinese blog at elasticstack.blog.csdn.net .

    liu-xiao-guo

Abstract

A detailed introduction to semantic search in Elasticsearch and its latest developments. Concrete examples show how to use vector search (both sparse and dense vectors) for semantic search, how to easily create agents with Elastic AI Agent Builder, and how to integrate agents using the MCP and A2A protocols.

Details

Elasticsearch is one of the most widely used search engines in the world. With the development of artificial intelligence technology, Elasticsearch has introduced vector search capabilities (including dense vectors and sparse vectors) to support more powerful semantic search. More importantly, Elasticsearch provides hybrid search (BM25 + vector search) capabilities. Elastic's hybrid search combines traditional keyword-based search (including flexible query syntax, precise keyword matching, and relevance scoring) with semantic understanding based on vector similarity, and supports a variety of reranking techniques. This collaborative approach can significantly improve the relevance of search results, enabling queries to not only match keywords but also understand user intent, thereby locating content more accurately. It is of great significance to use a data platform with distributed search, retrieval, and reranking capabilities as the core context retrieval engine. This topic will use vivid and concrete examples to deeply demonstrate how to leverage sparse vectors and dense vectors to achieve high-quality semantic search. Use the latest JINA embedding models to implement multimodal queries. In the latest Elastic release, Elastic launched the AI Agent Builder, enabling developers to quickly create their own Agents within minutes. Agent Builder provides a chat-based interactive interface, allowing users to create and manage agents, tools, and skills directly in Kibana. At the same time, Elastic also provides a Workflows automation engine, offering stable and reliable scripted automation capabilities for simple tasks, and AI-driven intelligent automation for complex scenarios that require reasoning. Developers can build tools in Kibana based on ES|QL, MCP, Workflows, and Index Search, and can also extend capabilities for specific search scenarios through custom skills. Users can query, correlate, and analyze data in Elastic through natural language, achieving true “conversation with data.”

Kibana has built-in MCP and A2A servers, programmable APIs, and a series of pre-built system tools for querying and exploring Elasticsearch indexes, and automatically generating ES|QL queries from natural language. Agent Builder also allows developers to create custom tools and use the richly expressive ES|QL query syntax to direct and shape the context data returned to the agent. Combined with Elasticsearch’s powerful aggregation, filtering, analysis, and hybrid search capabilities, Elastic provides modern AI agents with a rich and trustworthy contextual foundation, thereby building the next-generation intelligent search and analysis platform with contextual awareness.