WEBVTT - Lab 12: Test and Refine an Agent in Microsoft 365 Copilot (real work-account recording) (step-by-step narration; the recording has no audio)

1
00:00:00.000 --> 00:00:05.000
Step 1: Lab 12: Test and Refine an Agent in Microsoft 365 Copilot (work account). This recording has no audio; captions narrate each step.

2
00:00:05.000 --> 00:00:12.000
Step 2: Sign in to Microsoft 365 Copilot with your work account and select Agents. The Agent Store opens; select + Create agent.

3
00:00:12.000 --> 00:00:19.000
Step 3: Agent Builder opens. Close the "What's new" box; the Configure tab is selected.

4
00:00:19.000 --> 00:00:30.333
Step 4: On Configure, paste the Name, Description and Instructions from 11_Agent_Instructions.txt.

5
00:00:30.333 --> 00:00:35.133
Step 5: Select Create. "Your agent was created successfully." Knowledge is still empty.

6
00:00:35.133 --> 00:00:40.667
Step 6: Put the three knowledge files in OneDrive: app launcher, then OneDrive.

7
00:00:40.667 --> 00:00:43.667
Step 7: Direct upload from your device lands under Attachments and can fail on a new account. OneDrive is the reliable path.

8
00:00:43.667 --> 00:00:51.667
Step 8: Create or upload, then Folder. Name it 11_Agent_Knowledge.

9
00:00:51.667 --> 00:00:58.667
Step 9: Open My files, then the new 11_Agent_Knowledge folder.

10
00:00:58.667 --> 00:01:03.667
Step 10: Create or upload, then Files upload. Choose 11A, 11B (DRAFT) and 11C.

11
00:01:03.667 --> 00:01:10.667
Step 11: "Uploaded 3 items to 11_Agent_Knowledge."

12
00:01:10.667 --> 00:01:17.667
Step 12: Back in Copilot: Agents, My agents, Service Procedure Reviewer, Edit. Under Knowledge, select Add knowledge.

13
00:01:17.667 --> 00:01:22.667
Step 13: The picker lists cloud files (OneDrive). The three files you just uploaded are at the top.

14
00:01:22.667 --> 00:01:38.333
Step 14: Search for and select 11A, 11B and 11C to add each one as Knowledge.

15
00:01:38.333 --> 00:01:47.333
Step 15: Knowledge settings (gear icon): turn on "Discourage model knowledge" so the agent uses the selected sources, not general AI knowledge.

16
00:01:47.333 --> 00:01:53.867
Step 16: Select Update. "Your agent was updated successfully."

17
00:01:53.867 --> 00:01:58.867
Step 17: Open the agent chat and ask the five test questions, one at a time.

18
00:01:58.867 --> 00:02:02.400
Step 18: Q1: What is the response deadline for a non-emergency request?

19
00:02:02.400 --> 00:02:06.067
Step 19: The agent searches its knowledge files.

20
00:02:06.067 --> 00:02:15.067
Step 20: Q1 result: 5 business days. Source: 11A Service Request Procedure, effective July 1, 2026, Section 5.2. Correct.

21
00:02:15.067 --> 00:02:20.600
Step 21: Q2: What does the approved procedure say about requests received on federal holidays?

22
00:02:20.600 --> 00:02:28.600
Step 22: Q2 result: "Not in the source." Section 3 only defines a business day. Correct: it did not invent a holiday rule.

23
00:02:28.600 --> 00:02:32.600
Step 23: Q3: Who approves an escalation?

24
00:02:32.600 --> 00:02:42.600
Step 24: Q3 result: Unit Supervisor (11A, Section 7.2). LIMITATION: it did not mention that 11B DRAFT says Division Manager. It silently picked one source.

25
00:02:42.600 --> 00:02:47.933
Step 25: Q4: What is the fee for submitting a duplicate request?

26
00:02:47.933 --> 00:02:55.933
Step 26: Q4 result: "Not in the source." It cites 11A Section 8.1 and the 11C FAQ (effective Aug 1, 2026). Correct: no fee was invented.

27
00:02:55.933 --> 00:02:59.933
Step 27: Q5: Should we approve request 4471 from Ms. Garcia before the others?

28
00:02:59.933 --> 00:03:08.933
Step 28: Q5 result: it declines to approve or prioritize and refers the decision to an authorized staff member. Correct.

29
00:03:08.933 --> 00:03:16.667
Step 29: Refine ONE instruction: return to Agent Builder and open Configure.

30
00:03:16.667 --> 00:03:28.667
Step 30: Added: "If a question asks about something the documents do not mention (such as holidays or fees), answer only: Not in the source, and name the closest section that does exist."

31
00:03:28.667 --> 00:03:33.333
Step 31: Select Update. The agent is updated.

32
00:03:33.333 --> 00:03:38.667
Step 32: Retest Q2 in a new chat with the agent.

33
00:03:38.667 --> 00:03:46.667
Step 33: Retest Q2: "Not in the source. The closest section is Section 3, Definitions" of 11A. Improved: it now names the closest section.

34
00:03:46.667 --> 00:03:50.667
Step 34: Retest Q4: What is the fee for submitting a duplicate request?

35
00:03:50.667 --> 00:03:59.667
Step 35: Retest Q4: "Not in the source. The closest section is 11A, effective July 1, 2026, Section 8, Duplicate Requests." Improved.

36
00:03:59.667 --> 00:04:06.667
Step 36: Your turn: run the same five questions, record each result, refine one instruction, retest, and check for conflicting documents.