mirror of
https://github.com/tiddly-gittly/TidGi-Desktop.git
synced 2026-03-23 15:20:42 -07:00
64cc965a4a
* Add wiki tiddler attachment support to agent chat Implements the ability to attach wiki tiddlers to agent chat messages. Updates the UI to allow selection of tiddlers from active wiki workspaces, fetches and renders tiddler content as plain text, and appends it to the user message sent to the AI. Includes e2e tests, updates to store actions, service interfaces, and prompt concatenation logic to support this feature. * fix: callback not useCallback cause autocomplete panel flash * Add wiki tiddler attachments to message bubbles Message bubbles now display attached wiki tiddlers as clickable chips, allowing users to navigate directly to the referenced tiddler in the appropriate workspace. Metadata handling and persistence for wiki tiddlers has been updated to include workspaceId, and tests have been added to verify the new UI behavior. The chat view also now closes the TiddlyWiki sidebar for better focus when navigating from a selection. * Support split view navigation for wiki tiddler attachments Adds isSplitView prop to ChatTabContent and related components to distinguish between split view and normal tab modes. Wiki tiddler attachment navigation now uses a different strategy in split view, opening tiddlers directly in the browser view. Updates types and tests to reflect the new behavior, and improves robustness of response handling in several places. * docs: move to .github/instructions/testing.instructions.md * test: view loading slow on mac * refactor(e2e): move wiki load steps to Background in talkWithAI.feature; remove all sidebar close delays and polling, only set state when TiddlyWiki is ready; clean up code and logs for sidebar auto-close in split view * docs: make test inst shorter * lint * refactor(view): slim ViewService, move menu to separate file, orchestrate view logic in WorkspaceViewService, update all callers, fix lint floating promise, all unit and e2e tests pass * fix: add data-testid to attachment listbox for E2E test - Add slotProps to MUI Autocomplete to ensure attachment-listbox is rendered with correct test-id - Fix E2E test timeout when waiting for attachment listbox element * lint * put 'Talk with AI' menu on top and attachment i18n Introduce a reusable createTalkWithAIMenuItems helper to build "Talk with AI" menu entries (default agent + other agents submenu) and integrate it into workspace menu generation. Add new i18n keys for Agent.Attachment and WikiEmbed across locales and update UI to use translation keys (remove hardcoded fallback strings). Improve chat input/attachment behavior: expose a test-id for the attachment listbox, use i18n for labels/placeholders, and tweak input component wiring. Fix Cucumber step handling by normalizing expected newline sequences and safely handling empty message content. Also adjust memo deps in SortableWorkspaceSelectorButton to include id. * feat: enhance AI interaction in workspace context menu with local trigger support * feat: add tool approval and timeout settings - Introduced ToolApprovalConfig and related types for managing tool execution approvals. - Implemented WebFetch and ZxScript tools for fetching web content and executing scripts, respectively. - Added token estimation utilities for context window management. - Enhanced ModelInfo interface with context window size and max output tokens. - Created API Retry Utility for handling transient failures with exponential backoff. - Updated AIAgent preferences section to include Tool Approval & Timeout Settings dialog. - Developed ToolApprovalSettingsDialog for configuring tool-specific approval rules and retry settings. - Modified vitest configuration to support aliasing for easier imports and stubbing. * Refactor agent instance tools and services for improved modularity and maintainability - Extracted type definitions and tool registry from defineTool.ts into separate files (defineToolTypes.ts, toolRegistry.ts) to reduce file size and enhance importability. - Implemented a retry mechanism in ExternalAPIService for stream creation to handle transient failures. - Updated ToolApprovalSettingsDialog to persist settings using localStorage instead of a preference service. - Created agentMessagePersistence.ts and agentRepository.ts to manage agent message and instance CRUD operations, reducing the size of AgentInstanceService. - Added a progress tracker document (AgentTODO.md) for the ongoing enhancement plan of the TidGi Agent. * feat: add AgentSwitcher component for agent definition switching - Implemented AgentSwitcher component with dropdown functionality for selecting agent definitions. - Integrated loading of agent definitions on dropdown open. - Added visual feedback for current selection and disabled state. feat: create ToolResultRenderer for generic tool result messages - Developed ToolResultRenderer to handle rendering of <functions_result> messages. - Included collapsible parameters and result display with error handling. - Added truncation for long results in collapsed view. test: add comprehensive tests for MessageRenderer components - Implemented tests for AskQuestionRenderer, ToolResultRenderer, ToolApprovalRenderer, and BaseMessageRenderer. - Ensured proper rendering and functionality for various message types and states. - Included pattern routing tests for MessageRenderer. feat: introduce TurnActionBar for action management in agent turns - Created TurnActionBar component for managing actions like rollback, retry, delete, and copy. - Integrated visual feedback for file changes and rollback status. - Added functionality for copying agent responses and full conversation to clipboard. feat: implement askQuestionPending for managing user responses - Developed infrastructure for handling pending ask-question requests. - Implemented promise-based blocking until user responds to agent questions. - Added timeout handling for ask-question requests. * feat: Implement background task management for agent instances - Added functionality to restore heartbeat timers and alarms for active agents upon service initialization. - Introduced methods to retrieve active background tasks and cancel them via the UI. - Enhanced alarm clock tool to persist alarm data in the database, ensuring alarms survive app restarts. - Updated agent instance schema to include scheduled alarm data. - Modified prompt concatenation logic to support context window size for message history. - Removed system prompt parameter from model parameters schema and related components. - Improved UI to display and manage background tasks, including heartbeat and alarm details. * feat: Implement Scheduled Tasks Management and Background Task Settings - Add feature for managing scheduled tasks for agents, including viewing, adding, and editing tasks. - Create tests for agent repository and background task settings APIs. - Introduce ScheduledTaskManager for unified scheduling of tasks with interval, at, and cron schedules. - Implement edit-agent-definition tool for modifying agent configurations, including heartbeat and prompt settings. - Ensure tasks persist across app restarts and respect active hours filtering. * Update wiki * fix(security): harden htmlToText against XSS via encoded tags - Use tolerant script/style regex that handles </script > with spaces - Stop decoding </> entities to prevent reintroducing HTML tags - Decode & last to avoid double-unescaping (&lt; < <) - Fixes all 4 CodeQL findings (bad filtering, incomplete sanitization, double escaping, incomplete multi-character sanitization) * fix: lint and format errors (eslint naming, dprint formatting, stub classes) * fix: exclude tidgi.config.json from template copy and fix log marker pattern - Skip tidgi.config.json when copying wiki template to prevent overriding workspace name - Change waitForLogMarker default pattern from 'wiki-' to '*' to match actual log filenames - Filter tidgi.config.json in e2e step too (fs.copy in wiki creation step) * perf(e2e): merge 5 scheduledTask scenarios into 2 to reduce app restarts * perf(e2e): merge crossWindowSync and streamingStatus scenarios to reduce CI time * ci: increase test timeout to 20min for larger e2e scenario count * perf(e2e): merge agent scenarios and enable parallel on CI - Merge agent.feature wiki-search + wiki-operation into one scenario - Merge agent.feature create-agent-from-newtab + create-agent-from-fallback into one - Enable cucumber parallel: 2 on CI (7GB RAM, dynamic ports for mock servers) - Total scenarios: 66 -> 61 * fix: cleanup MCP client processes when deleting or closing agent * ci: disable parallel (CPU contention), increase timeout to 30min for 61 scenarios * perf(e2e): merge 3 preference background-task scenarios into 1, increase timeout to 45min - Preference alarm/heartbeat CRUD merged into single scenario (saves 2 app restarts) - Total scenarios: 59 (was 66) - CI timeout 45min for the expanded e2e suite * fix: restructure e2e scenarios, fix subscription leak and debounce cleanup - Restructure agent/talkWithAI/streamingStatus feature files for reliability - Fix talkWithAI scenarios missing mock server startup and app launch - Remove duplicate subscription in handleSwitchAgent (useEffect already handles it) - Clean up debounced update functions when agent is deleted/closed - Use agentId:messageId key for debounced functions to enable per-agent cleanup * fix(e2e): remove :last-child selectors broken by TurnActionBar, fix tab/dialog selectors, increase CI timeout to 25min * fix: use const for non-reassigned variable (lint) * fix(e2e): fix close-all-tabs opacity issue, scheduledTask undefined steps, MUI multiline textarea targeting * fix(e2e): use tab-list-dropdown to close all tabs, fix selector for actual TabListDropdown component * fix(e2e): add timing waits for BrowserView repositioning and git log UI refresh * fix(e2e): make 'should not see' step wait for element to disappear instead of instant check * fix(e2e): increase executeInBrowserView default timeout from 500ms to 2000ms
185 lines
19 KiB
Gherkin
185 lines
19 KiB
Gherkin
Feature: Vector Search - Embedding Generation and Semantic Search
|
||
As a user
|
||
I want to use vector database to perform semantic search in my wiki
|
||
So that I can find relevant content based on meaning rather than exact keywords
|
||
|
||
Background:
|
||
Given I add test ai settings
|
||
|
||
@vectorSearch @mockOpenAI
|
||
Scenario: Agent workflow - Create notes, update embeddings, then search
|
||
Given I have started the mock OpenAI server
|
||
| response | stream | embedding |
|
||
| <tool_use name="wiki-operation">{"workspaceName":"wiki","operation":"wiki-add-tiddler","title":"AI Agent Guide","text":"智能体是一种可以执行任务的AI系统，它可以使用工具、搜索信息并与用户交互。"}</tool_use> | false | |
|
||
| 已成功在工作区 wiki 中创建条目 "AI Agent Guide"。 | false | |
|
||
| <tool_use name="wiki-operation">{"workspaceName":"wiki","operation":"wiki-add-tiddler","title":"Vector Database Tutorial","text":"向量数据库用于存储和检索高维向量数据，支持语义搜索和相似度匹配。"}</tool_use> | false | |
|
||
| 已成功在工作区 wiki 中创建条目 "Vector Database Tutorial"。 | false | |
|
||
| <tool_use name="wiki-update-embeddings">{"workspaceName":"wiki","forceUpdate":false}</tool_use> | false | |
|
||
| | false | note1 |
|
||
| | false | note2 |
|
||
| 已成功为工作区 wiki 生成向量嵌入索引。总计2个笔记，2个嵌入向量。 | false | |
|
||
| <tool_use name="wiki-search">{"workspaceName":"wiki","searchType":"vector","query":"如何使用AI智能体","limit":5,"threshold":0.7}</tool_use> | false | |
|
||
| | false | query-note1 |
|
||
| 根据向量搜索结果，在工作区 wiki 中找到以下相关内容：\n\n**Tiddler: AI Agent Guide** (Similarity: 95.0%)\n这篇笔记介绍了AI智能体的基本概念和使用方法。 | false | |
|
||
# Launch application after mock server is ready
|
||
Then I launch the TidGi application
|
||
And I wait for the page to load completely
|
||
And I should see a "page body" element with selector "body"
|
||
# Ensure we are in the agent workspace (not wiki workspace) for agent interaction
|
||
When I click on an "agent workspace button" element with selector "[data-testid='workspace-agent']"
|
||
And I should see a "new tab button" element with selector "[data-tab-id='new-tab-button']"
|
||
# Step 1: Open agent chat interface
|
||
When I click on a "new tab button" element with selector "[data-tab-id='new-tab-button']"
|
||
And I should see a "search interface" element with selector ".aa-Autocomplete"
|
||
When I click on a "search input box" element with selector ".aa-Input"
|
||
And I should see an "autocomplete panel" element with selector ".aa-Panel"
|
||
When I click on an "agent suggestion" element with selector '[data-autocomplete-source-id="agentsSource"] .aa-ItemWrapper'
|
||
# Step 2: Create first note
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "在 wiki 工作区创建一个名为 AI Agent Guide 的笔记，内容是：智能体是一种可以执行任务的AI系统，它可以使用工具、搜索信息并与用户交互。" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 4 messages in chat history
|
||
# Step 3: Create second note
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "再创建一个名为 Vector Database Tutorial 的笔记，内容是：向量数据库用于存储和检索高维向量数据，支持语义搜索和相似度匹配。" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 8 messages in chat history
|
||
# Step 4: Update vector embeddings using agent tool
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "为 wiki 工作区更新向量索引" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 12 messages in chat history
|
||
# Step 5: Perform vector search
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "使用向量搜索在 wiki 中查找关于如何使用AI智能体的内容" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 16 messages in chat history
|
||
# Verify the last message contains vector search results
|
||
And I should see "search result in last message" elements with selectors:
|
||
| element description | selector |
|
||
| search result in last message| [data-testid='message-bubble']:has-text('Tiddler: AI Agent Guide') |
|
||
|
||
@vectorSearch @mockOpenAI
|
||
Scenario: UI workflow - Generate embeddings via preferences, then search
|
||
Given I have started the mock OpenAI server
|
||
| response | stream | embedding |
|
||
| <tool_use name="wiki-operation">{"workspaceName":"wiki","operation":"wiki-add-tiddler","title":"Machine Learning Basics","text":"机器学习是人工智能的一个分支，通过算法让计算机从数据中学习规律。"}</tool_use> | false | |
|
||
| 已成功在工作区 wiki 中创建条目 "Machine Learning Basics"。 | false | |
|
||
| | false | note3 |
|
||
| <tool_use name="wiki-search">{"workspaceName":"wiki","searchType":"vector","query":"机器学习","limit":5,"threshold":0.7}</tool_use> | false | |
|
||
| | false | query-note3 |
|
||
| 根据向量搜索结果，在工作区 wiki 中找到以下相关内容：\n\n**Tiddler: Machine Learning Basics** (Similarity: 98.0%)\n这篇笔记介绍了机器学习的基本概念。 | false | |
|
||
# Launch application after mock server is ready
|
||
Then I launch the TidGi application
|
||
And I wait for the page to load completely
|
||
And I should see a "page body" element with selector "body"
|
||
# Ensure we are in the agent workspace (not wiki workspace) for agent interaction
|
||
When I click on an "agent workspace button" element with selector "[data-testid='workspace-agent']"
|
||
And I should see a "new tab button" element with selector "[data-tab-id='new-tab-button']"
|
||
# Step 1: Create a test note via agent
|
||
When I click on "new tab button and create default agent button" elements with selectors:
|
||
| element description | selector |
|
||
| new tab button | [data-tab-id='new-tab-button'] |
|
||
| create default agent button| [data-testid='create-default-agent-button'] |
|
||
And I should see a "message input box" element with selector "[data-testid='agent-message-input']"
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "在 wiki 工作区创建一个名为 Machine Learning Basics 的笔记，内容是：机器学习是人工智能的一个分支，通过算法让计算机从数据中学习规律。" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 4 messages in chat history
|
||
# Step 2: Open preferences and navigate to Search section to generate embeddings
|
||
When I click on a "settings button" element with selector "#open-preferences-button"
|
||
When I switch to "preferences" window
|
||
When I click on a "search section" element with selector "[data-testid='preference-section-search']"
|
||
# Wait for workspace list to load
|
||
# The Search.tsx renders workspace cards with name, status, and buttons
|
||
And I should see a "wiki workspace card" element with selector "*:has-text('wiki')"
|
||
# Click the generate button - use button text "生成" instead of data-testid
|
||
# The button shows "生成" for initial generation, "更新嵌入" after generation
|
||
When I click on a "generate button with text" element with selector "button:has-text('生成')"
|
||
# Verify generation completed with detailed status information
|
||
# Should show: workspace name, embedding count, note count, last updated time and action buttons
|
||
Then I should see "workspace name in status and embedding count status and embedding word and last updated label and update button after generation and delete button after generation" elements with selectors:
|
||
| element description | selector |
|
||
| workspace name in status | *:has-text('wiki') |
|
||
| embedding count status | *:has-text('个笔记') |
|
||
| embedding word | *:has-text('嵌入') |
|
||
| last updated label | *:has-text('最后更新') |
|
||
| update button after generation | button:has-text('更新嵌入') |
|
||
| delete button after generation | button:has-text('删除') |
|
||
# Close preferences
|
||
When I close "preferences" window
|
||
And I switch to "main" window
|
||
# Step 3: Perform vector search and verify results match agent workflow
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "使用向量搜索在 wiki 中查找关于机器学习的内容" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 8 messages in chat history
|
||
# Verify the last message contains vector search results
|
||
And I should see a "ML search result in last message" element with selector "[data-testid='message-bubble']:has-text('Tiddler: Machine Learning Basics')"
|
||
|
||
@vectorSearch @mockOpenAI
|
||
Scenario: Vector search with low similarity - No results below threshold, then lower threshold
|
||
Given I have started the mock OpenAI server
|
||
| response | stream | embedding |
|
||
| <tool_use name="wiki-operation">{"workspaceName":"wiki","operation":"wiki-add-tiddler","title":"AI Technology","text":"人工智能技术正在改变世界。"}</tool_use> | false | |
|
||
| 已成功在工作区 wiki 中创建条目 "AI Technology"。 | false | |
|
||
| <tool_use name="wiki-operation">{"workspaceName":"wiki","operation":"wiki-add-tiddler","title":"Machine Learning","text":"机器学习算法和应用。"}</tool_use> | false | |
|
||
| 已成功在工作区 wiki 中创建条目 "Machine Learning"。 | false | |
|
||
| <tool_use name="wiki-update-embeddings">{"workspaceName":"wiki","forceUpdate":false}</tool_use> | false | |
|
||
| | false | note4 |
|
||
| | false | note5 |
|
||
| 已成功为工作区 wiki 生成向量嵌入索引。总计2个笔记，2个嵌入向量。 | false | |
|
||
| <tool_use name="wiki-search">{"workspaceName":"wiki","searchType":"vector","query":"天气预报","limit":5,"threshold":0.7}</tool_use> | false | |
|
||
| | false | unrelated |
|
||
| 在Wiki工作空间"wiki"中未找到符合条件的向量搜索结果（相似度阈值：0.7）。 | false | |
|
||
| <tool_use name="wiki-search">{"workspaceName":"wiki","searchType":"vector","query":"天气预报","limit":5,"threshold":0.1}</tool_use> | false | |
|
||
| | false | unrelated |
|
||
| 根据向量搜索结果，在工作区 wiki 中找到以下相关内容：\n\n**Tiddler: AI Technology** (Similarity: 15.0%)\n低相似度结果。 | false | |
|
||
# Launch application after mock server is ready
|
||
Then I launch the TidGi application
|
||
And I wait for the page to load completely
|
||
And I should see a "page body" element with selector "body"
|
||
# Ensure we are in the agent workspace (not wiki workspace) for agent interaction
|
||
When I click on an "agent workspace button" element with selector "[data-testid='workspace-agent']"
|
||
And I should see a "new tab button" element with selector "[data-tab-id='new-tab-button']"
|
||
# Step 1: Open agent chat interface
|
||
When I click on "new tab button and create default agent button" elements with selectors:
|
||
| element description | selector |
|
||
| new tab button | [data-tab-id='new-tab-button'] |
|
||
| create default agent button| [data-testid='create-default-agent-button'] |
|
||
And I should see a "message input box" element with selector "[data-testid='agent-message-input']"
|
||
# Step 2: Create first note about AI
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "在 wiki 工作区创建一个名为 AI Technology 的笔记，内容是：人工智能技术正在改变世界。" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 4 messages in chat history
|
||
# Step 3: Create second note about ML
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "再创建一个名为 Machine Learning 的笔记，内容是：机器学习算法和应用。" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 8 messages in chat history
|
||
# Step 4: Update vector embeddings
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "为 wiki 工作区更新向量索引" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 12 messages in chat history
|
||
# Step 5: Search for unrelated content with high threshold (should find nothing)
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "使用向量搜索在 wiki 中查找关于天气预报的内容，阈值设为0.7" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 16 messages in chat history
|
||
# Verify the response contains "no results found" with threshold info
|
||
Then I should see "no results and threshold 0.7" elements with selectors:
|
||
| element description | selector |
|
||
| no results message | [data-testid='message-bubble']:has-text('未找到符合条件') |
|
||
| threshold 0.7 message | [data-testid='message-bubble']:has-text('0.7') |
|
||
# Step 6: Lower threshold and search again (should find low-similarity results)
|
||
When I click on a "message input textarea" element with selector "[data-testid='agent-message-input']"
|
||
When I type "再次搜索天气预报，但这次把阈值降低到0.1" in "chat input" element with selector "[data-testid='agent-message-input']"
|
||
And I press "Enter" key
|
||
Then I should see 20 messages in chat history
|
||
# Verify the response contains low-similarity result
|
||
Then I should see "AI Technology and low similarity" elements with selectors:
|
||
| element description | selector |
|
||
| AI Technology message | [data-testid='message-bubble']:has-text('Tiddler: AI Technology') |
|
||
| low similarity message | [data-testid='message-bubble']:has-text('15') |
|