Microsoft Introduces Semantic Search in Copilot App on Windows 11

Microsoft’s Copilot app on Windows 11 is set to incorporate semantic search, enabling users to locate files and images using natural language. This means you can describe what you’re searching for instead of remembering specific file names or keywords within the documents. Examples provided by Microsoft include queries such as “find images of bridges at sunset on my PC,” “locate my CV,” or “find the recipe for chicken tostadas.”

To experience this new feature, users require a Copilot+ PC and should download the latest version (1.25082.132.0 and above) of the Copilot app from the Microsoft Store. Currently, the semantic search is available in English, Simplified Chinese, French, German, Japanese, and Spanish.

Initially introduced for Windows Search, File Explorer, and the Settings app on Copilot+ PCs earlier this year, semantic search operates without an internet connection, relying on a local language AI model that runs on your device’s Neural Processing Unit (NPU). Microsoft assured that no data collected during this process would be transmitted to any other computer or to the company itself. It remains unclear if these conditions apply equally when the feature is implemented within the Copilot app. We have reached out to Microsoft for clarification on this matter.

To better understand what the Copilot app can access, retrieve, or read, users are advised to navigate to the Copilot Settings under Permission settings.

The updated Copilot app also introduces two additional features. The revamped homepage now showcases your recent applications, files, and conversations. For the moment, it pulls PNG, JPEG, SVG, PDF, DOCX, XLSX, CSS, JSON, and TXT files from your standard Windows “Recent” folder.

Furthermore, there’s a Vision update in the offing. By selecting an app from the “Get guided help” section of the homepage, a Vision session will scan your entire screen and walk you through the basics of the selected application.

These features are integrated into the latest version of the Copilot app for Copilot+ PCs; however, they are being gradually rolled out, meaning not all Insiders will receive the update immediately according to Microsoft.