When you drop a 60‑slide deck into Copilot Chat, it will now use the charts, diagrams, and screenshots inside to answer—not just the surrounding text. For engineers rolling Copilot out, this is the difference between “read me the bullets” and “actually look at the graph on slide 14.”
What’s new: Copilot Chat can interpret embedded images in Word (.docx), PowerPoint (.pptx), and PDFs, and use those visuals to ground its answers. The intent: extract insights from charts, diagrams, and screenshots so complex questions get more accurate, complete responses. This builds on earlier “ask about visuals” features inside Word/PowerPoint, but matters more in Chat where people toss mixed file types at a single prompt.
How it works (and likely constraints)
- Capability: Vision + OCR over embedded images inside supported files; Copilot blends visual findings with surrounding text for a single answer.
- Scope per roadmap: .docx, .pptx, .pdf; visuals like charts, diagrams, screenshots. Excel is not listed.
- Practical limits (what’s not stated, but to expect): quality drops with low‑resolution exports, tiny axis labels, color‑only encodings, or heavily stylized charts. Scanned PDFs will behave like images; success depends on OCR quality. Decorative stock images can add noise.
- Security/governance: This should follow existing file grounding/permissions in Microsoft 365. Exact admin toggles, auditing detail for visual parsing, or DLP behavior on images aren’t called out here.
What this changes in practice
- Document Q&A gets better on visuals‑heavy decks and PDFs. Instead of copying numbers from charts into the prompt, you can ask directly about trends, outliers, or steps shown in a screenshot.
- Prompting shifts from “summarize the text” to “compare the chart on slide 4 with the table on slide 5; explain the variance.” Expect fewer back‑and‑forths.
- For enablement: authoring quality matters. Teams that export slides at low resolution or paste blurry screenshots will get poorer answers. Charts with legends/labels and clear contrast help.
What I’d watch
- Accuracy on complex visuals: Sankeys, stacked 100 percent bars, multi‑axis charts, and heatmaps are easy to misread. I’d be surprised if this nails all edge cases.
- File/page limits and timeouts: large PDFs and long decks may hit processing caps or partial parses.
- Non‑English text inside images; handwriting in screenshots.
- Governance: whether Purview/DLP classifications on images are respected at parse time, and what shows up in audit logs. Open question until Microsoft documents it.
How to pilot it this week
- Take three real files from your org: a KPI deck with charts, a process PDF with screenshots, a Word doc with a flow diagram. Ask targeted questions that reference specific visuals (“Explain the change between Q2 and Q3 in the bar chart on slide 8; cite the numbers you used”).
- Compare answers to a manual read. Note misses that correlate with low‑res, tiny fonts, or ambiguous legends; feed that back as content authoring guidance.
- Update prompt playbooks to call out visuals explicitly (“Use both text and any charts/diagrams when answering; if uncertain, state assumptions”).
The takeaway: Copilot Chat treating embedded images as first‑class context is a meaningful step for real‑world documents. It won’t fix sloppy charts or blurry screenshots, but it reduces the “explain the picture to the bot” busywork and raises the ceiling for accurate Q&A—provided your inputs are clean and your governance keeps up.