GitVita
New repositoryImport repositoryNew issue
Your repositories Your dashboard Sign in

microsoft/TypeAgent

Public

mirrored from https://github.com/microsoft/TypeAgentAvailable

Watch0Fork0Star0
CodeCommitsIssuesPull requestsActionsInsightsSecurity
copilot/remove-deprecated-dependencies

Branches

Tags

  • No tags available.
0Branches0Tags
Go to file
Add file
Code

Clone

HTTPS

Download ZIP

TypeAgent/dotnet/autoShell.Tests

dotnet/autoShell.Tests/manual-test-natural-language.txt

702lines · modecode

RawDownload
Latest commit unavailable.
1// ============================================================================
2// autoShell Natural Language Test Commands
3// ============================================================================
4// These are the natural language phrases a user would say to the Desktop Agent.
5// They follow the .agr grammar files and should be translated by TypeAgent
6// into the corresponding autoShell JSON commands.
7//
8// Use these to test the full pipeline: user speech → grammar → JSON → autoShell
9//
10// Each test has:
11// - The natural language command
12// - An undo command (where applicable)
13// - EXPECTED: what should happen on screen
14//
15// CAUTION lines mark commands that affect connectivity or are hard to undo.
16// ============================================================================
17
18
19// =====================
20// 1. APPLICATION MANAGEMENT (desktopSchema.agr)
21// =====================
22
23// 1a. Launch programs (grammar: launch/open/start/run <Program>)
24open notepad
25// EXPECTED: Notepad opens
26
27start calculator
28// EXPECTED: Calculator opens
29
30launch file explorer
31// EXPECTED: File Explorer opens
32
33run edge
34// EXPECTED: Microsoft Edge opens
35
36// 1b. Close programs (grammar: close/quit/exit <Program>)
37close calculator
38// EXPECTED: Calculator closes
39
40exit file explorer
41// EXPECTED: File Explorer closes
42
43quit edge
44// EXPECTED: Edge closes
45
46// 1c. Edge case: launch unknown app
47open some app that definitely does not exist
48// EXPECTED: Nothing happens, no crash
49
50// 1d. Edge case: close app that isn't running
51close spotify
52// EXPECTED: Nothing happens, no crash
53
54
55// =====================
56// 2. WINDOW MANAGEMENT (desktopSchema.agr)
57// =====================
58
59// 2a. Maximize (grammar: maximize <Program> / make <Program> full screen)
60maximize notepad
61// EXPECTED: Notepad window fills the screen
62
63make notepad full screen
64// EXPECTED: Same as above (alternate phrasing)
65
66// 2b. Minimize (grammar: minimize <Program> / hide <Program>)
67minimize notepad
68// EXPECTED: Notepad minimizes to taskbar
69
70hide notepad
71// EXPECTED: Same as above
72
73// 2c. Switch to (grammar: switch to / focus on / go to <Program>)
74switch to notepad
75// EXPECTED: Notepad comes to foreground
76
77focus on notepad
78// EXPECTED: Same
79
80go to notepad
81// EXPECTED: Same
82
83// 2d. Tile (grammar: tile <left> and <right> / put <left> on the left and <right> on the right)
84open calculator
85tile notepad and calculator
86// EXPECTED: Notepad on left half, Calculator on right half
87
88put notepad on the left and calculator on the right
89// EXPECTED: Same as above
90
91// 2e. Clean up
92close calculator
93close notepad
94
95
96// =====================
97// 3. VOLUME CONTROL (desktopSchema.agr)
98// =====================
99
100// 3a. Set volume (grammar: set volume to <number> / volume <number>)
101set volume to 30 percent
102// EXPECTED: System volume drops to 30%
103
104volume 80
105// EXPECTED: Volume jumps to 80%
106
107// 3b. Mute/unmute
108mute
109// EXPECTED: Audio muted
110
111unmute
112// EXPECTED: Audio unmuted
113
114// 3c. Restore
115set volume to 25
116restore volume
117// EXPECTED: Volume goes to 25, then restore brings back 80 (previous)
118
119
120// =====================
121// 4. THEME & MODE (desktopSchema.agr)
122// =====================
123
124// 4a. Dark mode (grammar: set theme to dark / enable dark mode / switch to dark)
125set theme to dark
126// EXPECTED: System switches to dark mode; taskbar and apps go dark
127
128// 4b. Light mode (undo)
129enable light mode
130// EXPECTED: System switches back to light mode
131
132// 4c. Toggle
133toggle mode
134// EXPECTED: Flips from current to opposite
135
136// 4d. Undo toggle
137toggle theme mode
138// EXPECTED: Flips back
139
140// 4e. Alternate phrasings
141switch to dark mode
142// EXPECTED: Dark mode
143switch to light
144// EXPECTED: Light mode
145
146
147// =====================
148// 5. WIFI & NETWORK (desktopSchema.agr)
149// =====================
150
151// 5a. Enable/disable wifi (grammar: turn on/off wifi / enable/disable wifi)
152turn on wifi
153// EXPECTED: Wi-Fi adapter enabled
154
155// CAUTION: This will disconnect you if on Wi-Fi!
156// turn off wifi
157// EXPECTED: Wi-Fi adapter disabled
158
159// 5b. Connect/disconnect (grammar: connect to <ssid> wifi / disconnect from wifi)
160// connect to MyNetwork wifi
161// EXPECTED: "Info: Connecting to WiFi network: MyNetwork"
162
163disconnect from wifi
164// EXPECTED: "Info: Disconnected from WiFi"
165
166// 5c. Airplane mode
167// CAUTION: Kills all wireless!
168// enable airplane mode
169// EXPECTED: All radios off
170
171// turn off airplane mode
172// EXPECTED: Radios restored
173
174// 5d. Bluetooth (grammar: turn on/off bluetooth / enable/disable bluetooth / toggle bluetooth)
175turn on bluetooth
176// EXPECTED: Bluetooth enabled via registry
177
178turn off bluetooth
179// EXPECTED: Bluetooth disabled
180
181toggle bluetooth
182// EXPECTED: Defaults to enable (true)
183
184// 5e. Enable wifi alternate phrasing
185enable wifi
186// EXPECTED: Wi-Fi on
187
188turn on wi-fi
189// EXPECTED: Same (hyphenated variant)
190
191
192// =====================
193// 6. BRIGHTNESS (desktopSchema.agr)
194// =====================
195
196// 6a. Increase (grammar: increase brightness / make the screen brighter / brighten the screen)
197increase brightness
198// EXPECTED: Screen brightness goes up ~10%
199
200make the screen brighter
201// EXPECTED: Same
202
203brighten the screen
204// EXPECTED: Same
205
206// 6b. Decrease (undo)
207decrease brightness
208// EXPECTED: Screen brightness goes down ~10%
209
210make the screen dimmer
211// EXPECTED: Same
212
213dim the screen
214// EXPECTED: Same
215
216
217// =====================
218// 7. VIRTUAL DESKTOPS (desktopSchema.agr)
219// =====================
220
221// 7a. Create desktop
222create new desktop
223// EXPECTED: A new virtual desktop appears (check with Win+Tab)
224
225create desktop Work
226// EXPECTED: Desktop named "Work" created
227
228// 7b. Switch
229next desktop
230// EXPECTED: Switches to next virtual desktop
231
232previous desktop
233// EXPECTED: Switches back
234
235switch to desktop 1
236// EXPECTED: Goes to first desktop
237
238// 7c. Move window
239open notepad
240move notepad to desktop 2
241// EXPECTED: Notepad moves to desktop 2
242
243// 7d. Pin
244pin notepad to all desktops
245// EXPECTED: Notepad visible on every desktop
246
247// 7e. Clean up
248close notepad
249
250// 7f. Notifications (grammar in desktop management)
251show notifications
252// EXPECTED: Action Center opens
253
254toggle notifications
255// EXPECTED: Action Center toggles
256
257
258// =====================
259// 8. PERSONALIZATION (personalizationSchema.agr)
260// =====================
261
262// 8a. Transparency (grammar: enable/disable transparency / turn on/off transparency effects)
263enable transparency
264// EXPECTED: Window transparency effects on
265
266turn off transparency effects
267// EXPECTED: Transparency off
268
269// 8b. Title bar color (grammar: apply color to title bar / enable/disable title bar color)
270apply color to title bar
271// EXPECTED: Title bars show accent color
272
273disable title bar color
274// EXPECTED: Title bars go back to default
275
276// 8c. High contrast
277open high contrast settings
278// EXPECTED: High contrast settings page opens
279
280
281// =====================
282// 9. TASKBAR (taskbarSchema.agr)
283// =====================
284
285// 9a. Alignment (grammar: center/left align the taskbar / align taskbar to center/left)
286left align the taskbar
287// EXPECTED: Start menu and icons move to the left
288
289center the taskbar
290// EXPECTED: Back to center (undo)
291
292align taskbar to left
293// EXPECTED: Alternate phrasing, same result
294
295align taskbar to center
296// EXPECTED: Undo
297
298// 9b. Auto-hide (grammar: auto hide the taskbar / always show the taskbar)
299auto hide the taskbar
300// EXPECTED: Taskbar slides off screen, shows on hover
301
302always show the taskbar
303// EXPECTED: Taskbar stays visible (undo)
304
305hide the taskbar
306// EXPECTED: Same as auto-hide
307
308show the taskbar
309// EXPECTED: Undo
310
311// 9c. Task View button
312hide task view button
313// EXPECTED: Task View button disappears from taskbar
314
315show task view button
316// EXPECTED: Button reappears (undo)
317
318// 9d. Widgets button
319hide widgets button
320// EXPECTED: Widgets button disappears
321
322show widgets
323// EXPECTED: Reappears (undo)
324
325// 9e. Badges
326enable taskbar badges
327// EXPECTED: Notification badges show on taskbar icons
328
329disable taskbar badges
330// EXPECTED: Badges hidden (undo)
331
332// 9f. Seconds in clock
333show seconds in clock
334// EXPECTED: System clock shows HH:MM:SS
335
336hide seconds in clock
337// EXPECTED: Clock shows HH:MM only (undo)
338
339// 9g. Multi-monitor
340show taskbar on all monitors
341// EXPECTED: Taskbar appears on all displays
342
343show taskbar only on main monitor
344// EXPECTED: Other monitors lose taskbar (undo)
345
346
347// =====================
348// 10. MOUSE & INPUT (inputSchema.agr)
349// =====================
350
351// 10a. Cursor speed (grammar: set mouse speed to <number> / mouse speed <number>)
352set mouse speed to 15
353// EXPECTED: Mouse moves faster
354
355mouse speed 10
356// EXPECTED: Back to default (undo)
357
358change mouse sensitivity to 5
359// EXPECTED: Slow cursor
360
361adjust mouse speed to 10
362// EXPECTED: Back to default
363
364// 10b. Scroll lines (grammar: set scroll lines to <number> / scroll <number> lines)
365set scroll lines to 5
366// EXPECTED: More scrolling per wheel notch
367
368scroll 3 lines per notch
369// EXPECTED: Back to default (undo)
370
371// 10c. Primary button (grammar: set primary mouse button to left/right / swap mouse buttons)
372swap mouse buttons
373// EXPECTED: Left and right click swap — UNDO IMMEDIATELY!
374
375set primary mouse button to left
376// EXPECTED: Back to normal (undo)
377
378use left handed mouse
379// EXPECTED: Right button becomes primary
380
381use right handed mouse
382// EXPECTED: Back to normal (undo)
383
384// 10d. Pointer precision (grammar: enable/disable enhanced pointer precision / mouse acceleration)
385enable mouse acceleration
386// EXPECTED: Pointer precision (acceleration) on
387
388disable mouse acceleration
389// EXPECTED: Off (undo)
390
391turn off mouse acceleration
392// EXPECTED: Same
393
394// 10e. Cursor trail (grammar: enable/disable cursor trail / mouse trail / set cursor trail length to <number>)
395enable cursor trail
396// EXPECTED: Mouse leaves a trail of cursors
397
398set cursor trail length to 10
399// EXPECTED: Longer trail
400
401disable cursor trail
402// EXPECTED: Trail gone (undo)
403
404turn on mouse trail
405// EXPECTED: Alternate phrasing
406
407turn off cursor trail
408// EXPECTED: Undo
409
410// 10f. Pointer size (grammar: increase/decrease pointer size / bigger/smaller cursor)
411bigger cursor
412// EXPECTED: Opens mouse pointer settings
413
414smaller cursor
415// EXPECTED: Opens mouse pointer settings
416
417make pointer bigger
418// EXPECTED: Same
419
420// 10g. Pointer customization
421change mouse pointer style to black
422// EXPECTED: Opens mouse pointer settings
423
424// 10h. Touchpad
425enable touchpad
426// EXPECTED: Opens touchpad settings
427
428disable touchpad
429// EXPECTED: Opens touchpad settings
430
431set touchpad speed to 5
432// EXPECTED: Opens touchpad settings
433
434adjust touchpad sensitivity
435// EXPECTED: Opens touchpad settings
436
437
438// =====================
439// 11. DISPLAY SETTINGS (displaySchema.agr)
440// =====================
441
442// 11a. Night light (grammar: enable/disable night light / blue light filter)
443enable night light
444// EXPECTED: Night light schedule enabled
445
446disable night light
447// EXPECTED: Night light off (undo)
448
449turn on blue light filter
450// EXPECTED: Same as enable night light
451
452turn off night light
453// EXPECTED: Undo
454
455// 11b. Color temperature
456adjust color temperature
457// EXPECTED: Opens Night light settings
458
459warmer screen colors
460// EXPECTED: Opens settings (reduce blue light)
461
462cooler colors
463// EXPECTED: Opens settings (increase blue light)
464
465// 11c. Scaling (grammar: set display scaling to <size> / set zoom to <size>)
466set display scaling to 125 percent
467// EXPECTED: Opens display settings, targets 125%
468
469set zoom to 150
470// EXPECTED: Opens display settings, targets 150%
471
472// 11d. Orientation
473set screen orientation to portrait
474// EXPECTED: Opens display settings
475
476rotate screen to landscape
477// EXPECTED: Opens display settings
478
479// 11e. Rotation lock
480lock screen rotation
481// EXPECTED: Rotation locked
482
483unlock rotation
484// EXPECTED: Unlocked (undo)
485
486
487// =====================
488// 12. PRIVACY (privacySchema.agr)
489// =====================
490
491// 12a. Camera (grammar: allow/deny camera access / enable/disable camera / turn on/off camera)
492deny camera access
493// EXPECTED: App camera access denied
494
495allow camera access
496// EXPECTED: Restored (undo)
497
498turn off camera
499// EXPECTED: Same as deny
500
501enable camera
502// EXPECTED: Undo
503
504// 12b. Microphone
505deny microphone access
506// EXPECTED: App microphone access denied
507
508turn on microphone
509// EXPECTED: Restored (undo)
510
511disable microphone
512// EXPECTED: Same as deny
513
514allow microphone access
515// EXPECTED: Undo
516
517// 12c. Location
518disable location services
519// EXPECTED: Location access denied
520
521enable location
522// EXPECTED: Restored (undo)
523
524turn off location
525// EXPECTED: Deny
526
527allow location access
528// EXPECTED: Undo
529
530
531// =====================
532// 13. POWER (powerSchema.agr)
533// =====================
534
535// 13a. Battery saver (grammar: set battery saver to <number> / activate battery saver at <number>)
536set battery saver to 30 percent
537// EXPECTED: Battery saver activates at 30%
538
539turn on battery saver at 20
540// EXPECTED: Back to default (undo)
541
542activate battery saver at 50 percent
543// EXPECTED: Threshold set to 50%
544
545battery saver at 20
546// EXPECTED: Undo
547
548// 13b. Power mode — plugged in
549set power mode to best performance
550// EXPECTED: Opens power settings
551
552enable balanced mode
553// EXPECTED: Opens power settings
554
555enable power saver mode
556// EXPECTED: Opens power settings
557
558// 13c. Power mode — on battery
559set battery power mode to best power efficiency
560// EXPECTED: Opens power settings
561
562on battery use best performance
563// EXPECTED: Opens power settings
564
565
566// =====================
567// 14. SYSTEM SETTINGS (systemSchema.agr)
568// =====================
569
570// 14a. Metered connection
571enable metered connection
572// EXPECTED: Opens network status settings
573
574disable metered connection
575// EXPECTED: Opens settings
576
577// 14b. Game mode
578enable game mode
579// EXPECTED: Opens Game Mode settings
580
581open game mode settings
582// EXPECTED: Same
583
584// 14c. Accessibility
585enable narrator
586// EXPECTED: Narrator starts speaking — disable quickly!
587
588stop narrator
589// EXPECTED: Narrator stops (undo)
590
591start magnifier
592// EXPECTED: Screen magnifier starts
593
594disable magnifier
595// EXPECTED: Magnifier closes (undo)
596
597enable sticky keys
598// EXPECTED: Sticky Keys on
599
600turn off sticky keys
601// EXPECTED: Off (undo)
602
603enable filter keys
604// EXPECTED: Filter Keys on
605
606disable filter keys
607// EXPECTED: Off (undo)
608
609enable mono audio
610// EXPECTED: Stereo collapses to mono
611
612turn off mono audio
613// EXPECTED: Stereo restored (undo)
614
615// 14d. File explorer
616show file extensions
617// EXPECTED: File names show ".txt", ".docx", etc. in Explorer
618
619hide file extensions
620// EXPECTED: Extensions hidden (undo)
621
622show hidden files
623// EXPECTED: Hidden/system files visible in Explorer
624
625hide hidden files
626// EXPECTED: Hidden (undo)
627
628display file extensions
629// EXPECTED: Alternate phrasing
630
631// 14e. Time settings
632enable automatic time sync
633// EXPECTED: Opens time settings / enables auto-sync
634
635enable automatic dst adjustment
636// EXPECTED: DST auto-adjustment on
637
638disable automatic dst
639// EXPECTED: Off (undo)
640
641automatically adjust for dst
642// EXPECTED: Alternate phrasing
643
644// 14f. Focus assist
645enable quiet hours
646// EXPECTED: Opens focus assist settings
647
648do not disturb
649// EXPECTED: Same
650
651open focus assist settings
652// EXPECTED: Same
653
654// 14g. Multi-monitor
655remember window locations
656// EXPECTED: Opens display settings
657
658minimize windows when monitor disconnects
659// EXPECTED: Opens display settings
660
661keep windows when monitor disconnects
662// EXPECTED: Opens display settings
663
664
665// =====================
666// 15. EDGE CASES & MIXED
667// =====================
668
669// 15a. Known program names from grammar
670open chrome
671open word
672open excel
673open powerpoint
674open outlook
675open visual studio code
676open paint
677open snipping tool
678open task manager
679open cmd
680open powershell
681// EXPECTED: Each opens the corresponding app (close them after testing)
682
683// 15b. Wildcard program names
684open my custom app
685launch something random
686// EXPECTED: Attempts to find and launch; may fail gracefully
687
688// 15c. Chained undo test
689set theme to dark
690enable transparency
691apply color to title bar
692left align the taskbar
693set mouse speed to 15
694show seconds in clock
695// Now undo everything:
696set theme to light
697disable transparency
698disable title bar color
699center the taskbar
700set mouse speed to 10
701hide seconds in clock
702// EXPECTED: All settings restored to defaults
703