Background: AI tools have dramatically boosted development efficiency, and large numbers of new web applications are shipped quickly — testing coverage pressure surges accordingly. Yet testing efficiency at most companies has not kept up, and traditional manual testing has become the bottleneck. This talk introduces a vision-enhanced Web UI testing framework implemented in Python.
· It examines the traditional UI-testing workflow based on CSS selectors, XPath, or DOM structure, and analyzes its fragility under page refactoring, dynamic rendering, and hallucinated output from AI-generated content.
· Based on Playwright + pytest, it demonstrates how to combine screenshots with natural-language descriptions to locate elements, plan actions, and complete assertions — making test cases closer to the human perspective and far less dependent on brittle selectors.
· It shows the complete AI-driven flow: AI automatically generates test cases, plans test steps, and finally outputs directly runnable Playwright scripts.
· For complex web applications, a Knowledge mechanism is introduced during testing to save successful examples and distill business rules and historical experience, enabling reuse of system knowledge.
Finally, it shows how to package the tool as MCP and Skill forms for easy invocation by AI tools, demonstrating its usability in agentic workflows.