Merge a027ccb25a into 94e6db10bb

2025-07-18 16:48:21 +00:00 · 2025-07-18 16:48:21 +00:00 · 2788990666
commit 2788990666
parent 94e6db10bb a027ccb25a
108 changed files with 26574 additions and 4197 deletions
--- a/.assets/perplexica-preview.gif
+++ b/.assets/perplexica-preview.gif
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@ -0,0 +1,145 @@
+# Project Overview
+
+Perplexica is an open-source AI-powered search engine that uses advanced machine learning to provide intelligent search results. It combines web search capabilities with LLM-based processing to understand and answer user questions, similar to Perplexity AI but fully open source.
+
+## Architecture
+
+The system works through these main steps:
+
+- User submits a query
+- The system determines if web search is needed
+- If needed, it searches the web using SearXNG
+- Results are ranked using embedding-based similarity search
+- LLMs are used to generate a comprehensive response with cited sources
+
+## Architecture Details
+
+### Technology Stack
+
+- **Frontend**: React, Next.js, Tailwind CSS
+- **Backend**: Node.js
+- **Database**: SQLite with Drizzle ORM
+- **AI/ML**: LangChain + LangGraph for orchestration
+- **Search**: SearXNG integration
+- **Content Processing**: Mozilla Readability, Cheerio, Playwright
+
+### Database (SQLite + Drizzle ORM)
+
+- Schema: `src/lib/db/schema.ts`
+- Tables: `messages`, `chats`, `systemPrompts`
+- Configuration: `drizzle.config.ts`
+- Local file: `data/db.sqlite`
+
+### AI/ML Stack
+
+- **LLM Providers**: OpenAI, Anthropic, Groq, Ollama, Gemini, DeepSeek, LM Studio
+- **Embeddings**: Xenova Transformers, similarity search (cosine/dot product)
+- **Agents**: `webSearchAgent`, `analyzerAgent`, `synthesizerAgent`, `taskManagerAgent`
+
+### External Services
+
+- **Search Engine**: SearXNG integration (`src/lib/searxng.ts`)
+- **Configuration**: TOML-based config file
+
+### Data Flow
+
+1. User query → Task Manager Agent
+2. Web Search Agent → SearXNG → Content extraction
+3. Analyzer Agent → Content processing + embedding
+4. Synthesizer Agent → LLM response generation
+5. Response with cited sources
+
+## Project Structure
+
+- `/src/app`: Next.js app directory with page components and API routes
+  - `/src/app/api`: API endpoints for search and LLM interactions
+- `/src/components`: Reusable UI components
+- `/src/lib`: Backend functionality
+  - `lib/search`: Search functionality and meta search agent
+  - `lib/db`: Database schema and operations
+  - `lib/providers`: LLM and embedding model integrations
+  - `lib/prompts`: Prompt templates for LLMs
+  - `lib/chains`: LangChain chains for various operations
+  - `lib/agents`: LangGraph agents for advanced processing
+  - `lib/utils`: Utility functions and types including web content retrieval and processing
+
+## Focus Modes
+
+Perplexica supports multiple specialized search modes:
+
+- All Mode: General web search
+- Local Research Mode: Research and interact with local files with citations
+- Chat Mode: Have a creative conversation
+- Academic Search Mode: For academic research
+- YouTube Search Mode: For video content
+- Wolfram Alpha Search Mode: For calculations and data analysis
+- Reddit Search Mode: For community discussions
+
+## Core Commands
+
+- **Development**: `npm run dev` (uses Turbopack for faster builds)
+- **Build**: `npm run build` (includes automatic DB push)
+- **Production**: `npm run start`
+- **Linting**: `npm run lint` (Next.js ESLint)
+- **Formatting**: `npm run format:write` (Prettier)
+- **Database**: `npm run db:push` (Drizzle migrations)
+
+## Configuration
+
+The application uses a `config.toml` file (created from `sample.config.toml`) for configuration, including:
+
+- API keys for various LLM providers
+- Database settings
+- Search engine configuration
+- Similarity measure settings
+
+## Common Tasks
+
+When working on this codebase, you might need to:
+
+- Add new API endpoints in `/src/app/api`
+- Modify UI components in `/src/components`
+- Extend search functionality in `/src/lib/search`
+- Add new LLM providers in `/src/lib/providers`
+- Update database schema in `/src/lib/db/schema.ts`
+- Create new prompt templates in `/src/lib/prompts`
+- Build new chains in `/src/lib/chains`
+- Implement new LangGraph agents in `/src/lib/agents`
+
+## AI Behavior Guidelines
+
+- Focus on factual, technical responses without unnecessary pleasantries
+- Avoid conciliatory language and apologies
+- Ask for clarification when requirements are unclear
+- Do not add dependencies unless explicitly requested
+- Only make changes relevant to the specific task
+- Do not create test files or run the application unless requested
+- Prioritize existing patterns and architectural decisions
+- Use the established component structure and styling patterns
+
+## Code Style & Standards
+
+### TypeScript Configuration
+
+- Strict mode enabled
+- ES2017 target
+- Path aliases: `@/*` → `src/*`
+- No test files (testing not implemented)
+
+### Formatting & Linting
+
+- ESLint: Next.js core web vitals rules
+- Prettier: Use `npm run format:write` before commits
+- Import style: Use `@/` prefix for internal imports
+
+### File Organization
+
+- Components: React functional components with TypeScript
+- API routes: Next.js App Router (`src/app/api/`)
+- Utilities: Grouped by domain (`src/lib/`)
+- Naming: camelCase for functions/variables, PascalCase for components
+
+### Error Handling
+
+- Use try/catch blocks for async operations
+- Return structured error responses from API routes
--- a/.gitignore
+++ b/.gitignore
@ -38,4 +38,6 @@ Thumbs.db
 db.sqlite
 /searxng

+# AI stuff for planning and implementation
+.ai/
 certificates
--- a/README.md
+++ b/README.md
@ -1,22 +1,6 @@
 # 🚀 Perplexica - An AI-powered search engine 🔎 <!-- omit in toc -->

-<div align="center" markdown="1">
-   <sup>Special thanks to:</sup>
-   <br>
-   <br>
-   <a href="https://www.warp.dev/perplexica">
-      <img alt="Warp sponsorship" width="400" src="https://github.com/user-attachments/assets/775dd593-9b5f-40f1-bf48-479faff4c27b">
-   </a>
-
-### [Warp, the AI Devtool that lives in your terminal](https://www.warp.dev/perplexica)
-
-[Available for MacOS, Linux, & Windows](https://www.warp.dev/perplexica)
-
-</div>
-
-<hr/>
-
-[![Discord](https://dcbadge.limes.pink/api/server/26aArMy8tT?style=flat)](https://discord.gg/26aArMy8tT)
+_This is a fork of [ItzCrazyKns/Perplexica](https://github.com/ItzCrazyKns/Perplexica) with additional features and improvements._

 ![preview](.assets/perplexica-screenshot.png?)

@ -31,11 +15,16 @@
  - [Ollama Connection Errors](#ollama-connection-errors)
 - [Using as a Search Engine](#using-as-a-search-engine)
 - [Using Perplexica's API](#using-perplexicas-api)
- [Expose Perplexica to a network](#expose-perplexica-to-network)
+- [Expose Perplexica to network](#expose-perplexica-to-network)
+  - [Running Behind a Reverse Proxy](#running-behind-a-reverse-proxy)
 - [One-Click Deployment](#one-click-deployment)
 - [Upcoming Features](#upcoming-features)
+- [Fork Improvements](#fork-improvements)
+  - [UI Improvements](#ui-improvements)
+  - [Search and Integration Enhancements](#search-and-integration-enhancements)
+  - [AI Functionality](#ai-functionality)
+  - [Bug Fixes](#bug-fixes)
 - [Support Us](#support-us)
-  - [Donations](#donations)
 - [Contribution](#contribution)
 - [Help and Support](#help-and-support)

@ -57,9 +46,10 @@ Want to know more about its architecture and how it works? You can read it [here
 - **Two Main Modes:**
  - **Copilot Mode:** (In development) Boosts search by generating different queries to find more relevant internet sources. Like normal search instead of just using the context by SearxNG, it visits the top matches and tries to find relevant sources to the user's query directly from the page.
  - **Normal Mode:** Processes your query and performs a web search.
- **Focus Modes:** Special modes to better answer specific types of questions. Perplexica currently has 6 focus modes:
+- **Focus Modes:** Special modes to better answer specific types of questions. Perplexica currently has 7 focus modes:
  - **All Mode:** Searches the entire web to find the best results.
-  - **Writing Assistant Mode:** Helpful for writing tasks that do not require searching the web.
+  - **Local Research Mode:** Research and interact with local files with citations.
+  - **Chat Mode:** Have a truly creative conversation without web search.
  - **Academic Search Mode:** Finds articles and papers, ideal for academic research.
  - **YouTube Search Mode:** Finds YouTube videos based on the search query.
  - **Wolfram Alpha Search Mode:** Answers queries that need calculations or data analysis using Wolfram Alpha.
@ -85,10 +75,10 @@ There are mainly 2 ways of installing Perplexica - With Docker, Without Docker.
 3. After cloning, navigate to the directory containing the project files.

 4. Rename the `sample.config.toml` file to `config.toml`. For Docker setups, you need only fill in the following fields:
-
   - `OPENAI`: Your OpenAI API key. **You only need to fill this if you wish to use OpenAI's models**.
   - `OLLAMA`: Your Ollama API URL. You should enter it as `http://host.docker.internal:PORT_NUMBER`. If you installed Ollama on port 11434, use `http://host.docker.internal:11434`. For other ports, adjust accordingly. **You need to fill this if you wish to use Ollama's models instead of OpenAI's**.
   - `GROQ`: Your Groq API key. **You only need to fill this if you wish to use Groq's hosted models**.
+   - `OPENROUTER`: Your OpenRouter API key. **You only need to fill this if you wish to use models via OpenRouter**.
   - `ANTHROPIC`: Your Anthropic API key. **You only need to fill this if you wish to use Anthropic models**.
   - `Gemini`: Your Gemini API key. **You only need to fill this if you wish to use Google's models**.
   - `DEEPSEEK`: Your Deepseek API key. **Only needed if you want Deepseek models.**
@ -126,7 +116,6 @@ If you're encountering an Ollama connection error, it is likely due to the backe

 1. **Check your Ollama API URL:** Ensure that the API URL is correctly set in the settings menu.
 2. **Update API URL Based on OS:**
-
   - **Windows:** Use `http://host.docker.internal:11434`
   - **Mac:** Use `http://host.docker.internal:11434`
   - **Linux:** Use `http://<private_ip_of_host>:11434`
@ -134,8 +123,7 @@ If you're encountering an Ollama connection error, it is likely due to the backe
   Adjust the port number if you're using a different one.

 3. **Linux Users - Expose Ollama to Network:**
-
-   - Inside `/etc/systemd/system/ollama.service`, you need to add `Environment="OLLAMA_HOST=0.0.0.0:11434"`. (Change the port number if you are using a different one.) Then reload the systemd manager configuration with `systemctl daemon-reload`, and restart Ollama by `systemctl restart ollama`. For more information see [Ollama docs](https://github.com/ollama/ollama/blob/main/docs/faq.md#setting-environment-variables-on-linux)
+   - Inside `/etc/systemd/system/ollama.service`, you need to add `Environment="OLLAMA_HOST=0.0.0.0"`. Then restart Ollama by `systemctl restart ollama`. For more information see [Ollama docs](https://github.com/ollama/ollama/blob/main/docs/faq.md#setting-environment-variables-on-linux)

   - Ensure that the port (default is 11434) is not blocked by your firewall.

@ -158,6 +146,38 @@ For more details, check out the full documentation [here](https://github.com/Itz

 Perplexica runs on Next.js and handles all API requests. It works right away on the same network and stays accessible even with port forwarding.

+### Running Behind a Reverse Proxy
+
+When running Perplexica behind a reverse proxy (like Nginx, Apache, or Traefik), follow these steps to ensure proper functionality:
+
+1. **Configure the BASE_URL setting**:
+   - In `config.toml`, set the `BASE_URL` parameter under the `[GENERAL]` section to your public-facing URL (e.g., `https://perplexica.yourdomain.com`)
+
+2. **Ensure proper headers forwarding**:
+   - Your reverse proxy should forward the following headers:
+     - `X-Forwarded-Host`
+     - `X-Forwarded-Proto`
+     - `X-Forwarded-Port` (if using non-standard ports)
+
+3. **Example Nginx configuration**:
+
+   ```nginx
+   server {
+     listen 80;
+     server_name perplexica.yourdomain.com;
+
+     location / {
+       proxy_pass http://localhost:3000;
+       proxy_set_header Host $host;
+       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+       proxy_set_header X-Forwarded-Proto $scheme;
+       proxy_set_header X-Forwarded-Host $host;
+     }
+   }
+   ```
+
+This ensures that OpenSearch descriptions, browser integrations, and all URLs work properly when accessing Perplexica through your reverse proxy.
+
 ## One-Click Deployment

 [![Deploy to Sealos](https://raw.githubusercontent.com/labring-actions/templates/main/Deploy-on-Sealos.svg)](https://usw.sealos.io/?openapp=system-template%3FtemplateName%3Dperplexica)
@ -174,24 +194,66 @@ Perplexica runs on Next.js and handles all API requests. It works right away on
 - [x] Adding Discover
 - [ ] Finalizing Copilot Mode

+## Fork Improvements
+
+This fork adds several enhancements to the original Perplexica project:
+
+### UI Improvements
+
+- ✅ Tabbed interface for message results
+- ✅ Added message editing capability
+- ✅ Ability to select AI models directly while chatting without opening settings
+- ✅ Change focus mode at any time during chat sessions
+- ✅ Auto-scrolling
+- ✅ Syntax highlighting for code blocks
+- ✅ Display search query with the response
+- ✅ Improved styling for all screen sizes
+- ✅ Added model statistics showing model name and response time
+- ✅ Shows progress during processing
+- ✅ Secures API keys by not showing them in the UI
+
+### Search and Integration Enhancements
+
+- ✅ OpenSearch support with dynamic XML generation
+  - Added BASE_URL config to support reverse proxy deployments
+  - Added autocomplete functionality proxied to SearxNG
+- ✅ Enhanced Reddit focus mode to work around SearxNG limitations
+- ✅ Enhanced Balance mode that uses a headless web browser to retrieve web content and use relevant excerpts to enhance responses
+- ✅ Adds Agent mode that uses the full content of web pages to answer queries and an agentic flow to intelligently answer complex queries with accuracy
+  - See the [README.md](docs/architecture/README.md) in the docs architecture directory for more info
+- ✅ Query-based settings override for browser search engine integration
+  - Automatically applies user's saved optimization mode and AI model preferences when accessing via URL with `q` parameter
+  - Enables seamless browser search bar integration with personalized settings
+
+### AI Functionality
+
+- ✅ True chat mode implementation (moved writing mode to local research mode)
+- ✅ Enhanced system prompts for more reliable and relevant results
+- ✅ Better parsing for reasoning models
+- ✅ User customizable context window for Ollama models
+- ✅ Toggle for automatic suggestions
+- ✅ Added support for latest Anthropic models
+- ✅ Adds support for multiple user-customizable system prompt enhancement and personas so you can tailor output to your needs
+- ✅ **Model Visibility Management**: Server administrators can hide specific models from the user interface and API responses
+  - Hide expensive models to prevent accidental usage and cost overruns
+  - Remove non-functional or problematic models from user selection
+  - Configurable via settings UI with collapsible provider interface for better organization
+  - API support with `include_hidden` parameter for administrative access
+
+### Bug Fixes
+
+- ✅ Improved history rewriting
+
 ## Support Us

 If you find Perplexica useful, consider giving us a star on GitHub. This helps more people discover Perplexica and supports the development of new features. Your support is greatly appreciated.

-### Donations
-
-We also accept donations to help sustain our project. If you would like to contribute, you can use the following options to donate. Thank you for your support!
-
-| Ethereum                                              |
-| ----------------------------------------------------- |
-| Address: `0xB025a84b2F269570Eb8D4b05DEdaA41D8525B6DD` |
-
 ## Contribution

 Perplexica is built on the idea that AI and large language models should be easy for everyone to use. If you find bugs or have ideas, please share them in via GitHub Issues. For more information on contributing to Perplexica you can read the [CONTRIBUTING.md](CONTRIBUTING.md) file to learn more about Perplexica and how you can contribute to it.

 ## Help and Support

-If you have any questions or feedback, please feel free to reach out to us. You can create an issue on GitHub or join our Discord server. There, you can connect with other users, share your experiences and reviews, and receive more personalized help. [Click here](https://discord.gg/EFwsmQDgAu) to join the Discord server. To discuss matters outside of regular support, feel free to contact me on Discord at `itzcrazykns`.
+If you have any questions or feedback, please feel free to reach out to us. You can create an issue on GitHub to get support or report bugs.

 Thank you for exploring Perplexica, the AI-powered search engine designed to enhance your search experience. We are constantly working to improve Perplexica and expand its capabilities. We value your feedback and contributions which help us make Perplexica even better. Don't forget to check back for updates and new features!
--- a/app.dockerfile
+++ b/app.dockerfile
@ -1,9 +1,10 @@
-FROM node:20.18.0-slim AS builder
+FROM --platform=linux/amd64 node:20-slim AS builder

 WORKDIR /home/perplexica

 COPY package.json yarn.lock ./
 RUN yarn install --frozen-lockfile --network-timeout 600000
+ENV NEXT_TELEMETRY_DISABLED=1

 COPY tsconfig.json next.config.mjs next-env.d.ts postcss.config.js drizzle.config.ts tailwind.config.ts ./
 COPY src ./src
@ -15,7 +16,9 @@ RUN yarn build
 RUN yarn add --dev @vercel/ncc
 RUN yarn ncc build ./src/lib/db/migrate.ts -o migrator

-FROM node:20.18.0-slim
+FROM --platform=linux/amd64 node:20-slim
+
+ENV NEXT_TELEMETRY_DISABLED=1

 WORKDIR /home/perplexica

@ -28,8 +31,14 @@ COPY drizzle ./drizzle
 COPY --from=builder /home/perplexica/migrator/build ./build
 COPY --from=builder /home/perplexica/migrator/index.js ./migrate.js

-RUN mkdir /home/perplexica/uploads
-
 COPY entrypoint.sh ./entrypoint.sh
-RUN chmod +x ./entrypoint.sh
+
+RUN mkdir /home/perplexica/uploads && \
+    chmod +x /home/perplexica/entrypoint.sh && \
+    npm install playwright && \
+    npx -y playwright install chromium --only-shell --with-deps && \
+    apt-get update && \
+    apt-get install -y procps && \
+    apt-get clean && rm -rf /var/lib/apt/lists/*
+
 CMD ["./entrypoint.sh"]
--- a/docker-compose.yaml
+++ b/docker-compose.yaml
@ -14,6 +14,8 @@ services:
    build:
      context: .
      dockerfile: app.dockerfile
+      platforms:
+        - linux/amd64
    environment:
      - SEARXNG_API_URL=http://searxng:8080
      - DATA_DIR=/home/perplexica
--- a/docs/API/SEARCH.md
+++ b/docs/API/SEARCH.md
@ -41,7 +41,6 @@ The API accepts a JSON object in the request body, where you define the focus mo
 ### Request Parameters

 - **`chatModel`** (object, optional): Defines the chat model to be used for the query. For model details you can send a GET request at `http://localhost:3000/api/models`. Make sure to use the key value (For example "gpt-4o-mini" instead of the display name "GPT 4 omni mini").
-
  - `provider`: Specifies the provider for the chat model (e.g., `openai`, `ollama`).
  - `name`: The specific model from the chosen provider (e.g., `gpt-4o-mini`).
  - Optional fields for custom OpenAI configuration:
@ -49,18 +48,15 @@ The API accepts a JSON object in the request body, where you define the focus mo
    - `customOpenAIKey`: The API key for a custom OpenAI instance.

 - **`embeddingModel`** (object, optional): Defines the embedding model for similarity-based searching. For model details you can send a GET request at `http://localhost:3000/api/models`. Make sure to use the key value (For example "text-embedding-3-large" instead of the display name "Text Embedding 3 Large").
-
  - `provider`: The provider for the embedding model (e.g., `openai`).
  - `name`: The specific embedding model (e.g., `text-embedding-3-large`).

 - **`focusMode`** (string, required): Specifies which focus mode to use. Available modes:
-
-  - `webSearch`, `academicSearch`, `writingAssistant`, `wolframAlphaSearch`, `youtubeSearch`, `redditSearch`.
+  - `webSearch`, `academicSearch`, `localResearch`, `chat`, `wolframAlphaSearch`, `youtubeSearch`, `redditSearch`.

 - **`optimizationMode`** (string, optional): Specifies the optimization mode to control the balance between performance and quality. Available modes:
-
-  - `speed`: Prioritize speed and return the fastest answer.
-  - `balanced`: Provide a balanced answer with good speed and reasonable quality.
+  - `speed`: Prioritize speed and get the quickest possible answer. Minimum effort retrieving web content. - Only uses SearXNG result previews.
+  - `agent`: Use an agentic workflow to answer complex multi-part questions. This mode requires a model that is trained for tool use.

 - **`query`** (string, required): The search query or question.

--- a/docs/architecture/README.md
+++ b/docs/architecture/README.md
@ -7,5 +7,12 @@ Perplexica's architecture consists of the following key components:
 3. **SearXNG**: A metadata search engine used by Perplexica to search the web for sources.
 4. **LLMs (Large Language Models)**: Utilized by agents and chains for tasks like understanding content, writing responses, and citing sources. Examples include Claude, GPTs, etc.
 5. **Embedding Models**: To improve the accuracy of search results, embedding models re-rank the results using similarity search algorithms such as cosine similarity and dot product distance.
+6. **Web Content**
+   - In Agent mode, the application uses an agentic workflow to answer complex multi-part questions
+     - The agent can use reasoning steps to provide comprehensive answers to complex questions
+     - Agent mode is experimental and may consume lots of tokens and take a long time to produce responses
+   - In Speed mode, the application only uses the preview content returned by SearXNG
+     - This content is provided by the search engines and contains minimal context from the actual web page
+     - This mode prioritizes quick responses over accuracy

 For a more detailed explanation of how these components work together, see [WORKING.md](https://github.com/ItzCrazyKns/Perplexica/tree/master/docs/architecture/WORKING.md).
--- a/docs/installation/configuration.md
+++ b/docs/installation/configuration.md
@ -0,0 +1,172 @@
+# Configuration Guide
+
+This guide covers all the configuration options available in Perplexica's `config.toml` file.
+
+## Configuration File Structure
+
+Perplexica uses a TOML configuration file to manage settings. Copy `sample.config.toml` to `config.toml` and modify it according to your needs.
+
+```bash
+cp sample.config.toml config.toml
+```
+
+## Configuration Sections
+
+### [GENERAL]
+
+General application settings.
+
+#### SIMILARITY_MEASURE
+
+- **Type**: String
+- **Options**: `"cosine"` or `"dot"`
+- **Default**: `"cosine"`
+- **Description**: The similarity measure used for embedding comparisons in search results ranking.
+
+#### KEEP_ALIVE
+
+- **Type**: String
+- **Default**: `"5m"`
+- **Description**: How long to keep Ollama models loaded into memory. Use time suffixes like `"5m"` for 5 minutes, `"1h"` for 1 hour, or `"-1m"` for indefinite.
+
+#### BASE_URL
+
+- **Type**: String
+- **Default**: `""` (empty)
+- **Description**: Optional base URL override. When set, overrides the detected URL for OpenSearch and other public URLs.
+
+#### HIDDEN_MODELS
+
+- **Type**: Array of Strings
+- **Default**: `[]` (empty array)
+- **Description**: Array of model names to hide from the user interface and API responses. Hidden models will not appear in model selection lists but can still be used if directly specified.
+- **Example**: `["gpt-4", "claude-3-opus", "expensive-model"]`
+- **Use Cases**:
+  - Hide expensive models to prevent accidental usage
+  - Remove models that don't work well with your configuration
+  - Simplify the UI by hiding unused models
+
+### [MODELS]
+
+Model provider configurations. Each provider has its own subsection.
+
+#### [MODELS.OPENAI]
+
+- **API_KEY**: Your OpenAI API key
+
+#### [MODELS.GROQ]
+
+- **API_KEY**: Your Groq API key
+
+#### [MODELS.ANTHROPIC]
+
+- **API_KEY**: Your Anthropic API key
+
+#### [MODELS.GEMINI]
+
+- **API_KEY**: Your Google Gemini API key
+
+#### [MODELS.CUSTOM_OPENAI]
+
+Configuration for OpenAI-compatible APIs (like LMStudio, vLLM, etc.)
+
+- **API_KEY**: API key for the custom endpoint
+- **API_URL**: Base URL for the OpenAI-compatible API
+- **MODEL_NAME**: Name of the model to use
+
+#### [MODELS.OLLAMA]
+
+- **API_URL**: Ollama server URL (e.g., `"http://host.docker.internal:11434"`)
+
+#### [MODELS.DEEPSEEK]
+
+- **API_KEY**: Your DeepSeek API key
+
+#### [MODELS.LM_STUDIO]
+
+- **API_URL**: LM Studio server URL (e.g., `"http://host.docker.internal:1234"`)
+
+### [API_ENDPOINTS]
+
+External service endpoints.
+
+#### SEARXNG
+
+- **Type**: String
+- **Description**: SearxNG API URL for web search functionality
+- **Example**: `"http://localhost:32768"`
+
+## Environment Variables
+
+Some configurations can also be set via environment variables, which take precedence over the config file:
+
+- `OPENAI_API_KEY`
+- `GROQ_API_KEY`
+- `ANTHROPIC_API_KEY`
+- `GEMINI_API_KEY`
+- And others following the pattern `{PROVIDER}_API_KEY`
+
+## Model Visibility Management
+
+The `HIDDEN_MODELS` setting allows server administrators to control which models are visible to users:
+
+### How It Works
+
+1. Models listed in `HIDDEN_MODELS` are filtered out of API responses
+2. The settings UI shows all models (including hidden ones) for management
+3. Hidden models can still be used if explicitly specified in API calls
+
+### Managing Hidden Models
+
+1. **Via Configuration File**: Edit the `HIDDEN_MODELS` array in `config.toml`
+2. **Via Settings UI**: Use the "Model Visibility" section in the settings page
+3. **Via API**: Use the `/api/config` endpoint to update the configuration
+
+### API Behavior
+
+- **Default**: `/api/models` returns only visible models
+- **Include Hidden**: `/api/models?include_hidden=true` returns all models (for admin use)
+
+## Security Considerations
+
+- Store API keys securely and never commit them to version control
+- Use environment variables for sensitive configuration in production
+- Regularly rotate API keys
+- Consider using `HIDDEN_MODELS` to prevent access to expensive or sensitive models
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Models not appearing**: Check if they're listed in `HIDDEN_MODELS`
+2. **API errors**: Verify API keys and URLs are correct
+3. **Ollama connection issues**: Ensure the Ollama server is running and accessible
+4. **SearxNG not working**: Verify the SearxNG endpoint is correct and accessible
+
+### Configuration Validation
+
+The application validates configuration on startup and will log errors for:
+
+- Invalid TOML syntax
+- Missing required fields
+- Invalid URLs or API endpoints
+- Unreachable services
+
+## Example Configuration
+
+```toml
+[GENERAL]
+SIMILARITY_MEASURE = "cosine"
+KEEP_ALIVE = "5m"
+BASE_URL = ""
+HIDDEN_MODELS = ["gpt-4", "claude-3-opus"]
+
+[MODELS.OPENAI]
+API_KEY = "sk-your-openai-key-here"
+
+[MODELS.OLLAMA]
+API_URL = "http://localhost:11434"
+
+[API_ENDPOINTS]
+SEARXNG = "http://localhost:32768"
+```
--- a/drizzle/0001_tricky_thor.sql
+++ b/drizzle/0001_tricky_thor.sql
@ -0,0 +1,8 @@
+CREATE TABLE IF NOT EXISTS `system_prompts` (
+	`id` text PRIMARY KEY NOT NULL,
+	`name` text NOT NULL,
+	`content` text NOT NULL,
+	`type` text DEFAULT 'system' NOT NULL,
+	`created_at` integer NOT NULL,
+	`updated_at` integer NOT NULL
+);
--- a/drizzle/meta/0001_snapshot.json
+++ b/drizzle/meta/0001_snapshot.json
@ -0,0 +1,169 @@
+{
+  "version": "6",
+  "dialect": "sqlite",
+  "id": "92ab35e3-541f-48ba-9a8d-ecb12c5c8a06",
+  "prevId": "ef3a044b-0f34-40b5-babb-2bb3a909ba27",
+  "tables": {
+    "chats": {
+      "name": "chats",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "text",
+          "primaryKey": true,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "title": {
+          "name": "title",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "createdAt": {
+          "name": "createdAt",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "focusMode": {
+          "name": "focusMode",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "files": {
+          "name": "files",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": false,
+          "autoincrement": false,
+          "default": "'[]'"
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {},
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "checkConstraints": {}
+    },
+    "messages": {
+      "name": "messages",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "integer",
+          "primaryKey": true,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "content": {
+          "name": "content",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "chatId": {
+          "name": "chatId",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "messageId": {
+          "name": "messageId",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "type": {
+          "name": "type",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": false,
+          "autoincrement": false
+        },
+        "metadata": {
+          "name": "metadata",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": false,
+          "autoincrement": false
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {},
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "checkConstraints": {}
+    },
+    "system_prompts": {
+      "name": "system_prompts",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "text",
+          "primaryKey": true,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "name": {
+          "name": "name",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "content": {
+          "name": "content",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "type": {
+          "name": "type",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false,
+          "default": "'system'"
+        },
+        "created_at": {
+          "name": "created_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "updated_at": {
+          "name": "updated_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {},
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "checkConstraints": {}
+    }
+  },
+  "views": {},
+  "enums": {},
+  "_meta": {
+    "schemas": {},
+    "tables": {},
+    "columns": {}
+  },
+  "internal": {
+    "indexes": {}
+  }
+}
--- a/drizzle/meta/_journal.json
+++ b/drizzle/meta/_journal.json
@ -8,6 +8,13 @@
      "when": 1748405503809,
      "tag": "0000_fuzzy_randall",
      "breakpoints": true
+    },
+    {
+      "idx": 1,
+      "version": "6",
+      "when": 1752728893879,
+      "tag": "0001_tricky_thor",
+      "breakpoints": true
    }
  ]
 }
--- a/next.config.mjs
+++ b/next.config.mjs
@ -8,7 +8,7 @@ const nextConfig = {
      },
    ],
  },
-  serverExternalPackages: ['pdf-parse'],
+  serverExternalPackages: ['pdf-parse', 'crawlee', 'playwright'],
 };

 export default nextConfig;
--- a/package-lock.json
+++ b/package-lock.json
--- a/package.json
+++ b/package.json
@ -4,43 +4,53 @@
  "license": "MIT",
  "author": "ItzCrazyKns",
  "scripts": {
-    "dev": "next dev",
+    "dev": "next dev --turbopack",
    "build": "npm run db:push && next build",
    "start": "next start",
    "lint": "next lint",
    "format:write": "prettier . --write",
-    "db:push": "drizzle-kit push"
+    "db:push": "drizzle-kit push",
+    "db:generate": "drizzle-kit generate"
  },
  "dependencies": {
    "@headlessui/react": "^2.2.0",
    "@iarna/toml": "^2.2.5",
    "@icons-pack/react-simple-icons": "^12.3.0",
-    "@langchain/anthropic": "^0.3.15",
-    "@langchain/community": "^0.3.36",
-    "@langchain/core": "^0.3.42",
-    "@langchain/google-genai": "^0.1.12",
-    "@langchain/openai": "^0.0.25",
+    "@langchain/anthropic": "^0.3.21",
+    "@langchain/community": "^0.3.45",
+    "@langchain/core": "^0.3.57",
+    "@langchain/google-genai": "^0.2.10",
+    "@langchain/groq": "^0.2.3",
+    "@langchain/langgraph": "^0.3.1",
+    "@langchain/ollama": "^0.2.0",
+    "@langchain/openai": "^0.5.12",
    "@langchain/textsplitters": "^0.1.0",
+    "@mozilla/readability": "^0.6.0",
    "@tailwindcss/typography": "^0.5.12",
+    "@types/react-syntax-highlighter": "^15.5.13",
    "@xenova/transformers": "^2.17.2",
    "axios": "^1.8.3",
    "better-sqlite3": "^11.9.1",
+    "cheerio": "^1.1.0",
    "clsx": "^2.1.0",
    "compute-cosine-similarity": "^1.1.0",
    "compute-dot": "^1.1.0",
    "drizzle-orm": "^0.40.1",
    "html-to-text": "^9.0.5",
+    "jsdom": "^26.1.0",
    "jspdf": "^3.0.1",
-    "langchain": "^0.1.30",
-    "lucide-react": "^0.363.0",
+    "langchain": "^0.3.26",
+    "lucide-react": "^0.525.0",
    "mammoth": "^1.9.1",
    "markdown-to-jsx": "^7.7.2",
    "next": "^15.2.2",
-    "next-themes": "^0.3.0",
+    "next-themes": "^0.4.6",
    "pdf-parse": "^1.1.1",
-    "react": "^18",
-    "react-dom": "^18",
-    "react-text-to-speech": "^0.14.5",
+    "playwright": "^1.52.0",
+    "react": "^19",
+    "react-dom": "^19",
+    "react-syntax-highlighter": "^15.6.1",
+    "react-text-to-speech": "^2.1.2",
    "react-textarea-autosize": "^8.5.3",
    "sonner": "^1.4.41",
    "tailwind-merge": "^2.2.2",
@ -51,11 +61,12 @@
  "devDependencies": {
    "@types/better-sqlite3": "^7.6.12",
    "@types/html-to-text": "^9.0.4",
+    "@types/jsdom": "^21.1.7",
    "@types/jspdf": "^2.0.0",
    "@types/node": "^20",
    "@types/pdf-parse": "^1.1.4",
-    "@types/react": "^18",
-    "@types/react-dom": "^18",
+    "@types/react": "^19",
+    "@types/react-dom": "^19",
    "autoprefixer": "^10.0.1",
    "drizzle-kit": "^0.30.5",
    "eslint": "^8",
--- a/sample.config.toml
+++ b/sample.config.toml
@ -1,6 +1,8 @@
 [GENERAL]
 SIMILARITY_MEASURE = "cosine" # "cosine" or "dot"
 KEEP_ALIVE = "5m" # How long to keep Ollama models loaded into memory. (Instead of using -1 use "-1m")
+BASE_URL = "" # Optional. When set, overrides detected URL for OpenSearch and other public URLs
+HIDDEN_MODELS = [] # Array of model names to hide from the user interface (e.g., ["gpt-4", "claude-3-opus"])

 [MODELS.OPENAI]
 API_KEY = ""
@ -8,6 +10,9 @@ API_KEY = ""
 [MODELS.GROQ]
 API_KEY = ""

+[MODELS.OPENROUTER]
+API_KEY = ""
+
 [MODELS.ANTHROPIC]
 API_KEY = ""

--- a/src/app/api/autocomplete/route.ts
+++ b/src/app/api/autocomplete/route.ts
@ -0,0 +1,78 @@
+import { NextResponse } from 'next/server';
+import { getSearxngApiEndpoint } from '@/lib/config';
+
+/**
+ * Proxies autocomplete requests to SearXNG
+ */
+export async function GET(request: Request) {
+  try {
+    // Get the query parameter from the request URL
+    const { searchParams } = new URL(request.url);
+    const query = searchParams.get('q');
+
+    // Check if query exists
+    if (!query) {
+      return new NextResponse(JSON.stringify([query || '', []]), {
+        headers: {
+          'Content-Type': 'application/x-suggestions+json',
+        },
+      });
+    }
+
+    // Get the SearXNG API endpoint
+    const searxngUrl = getSearxngApiEndpoint();
+
+    if (!searxngUrl) {
+      console.error('SearXNG API endpoint not configured');
+      return new NextResponse(JSON.stringify([query, []]), {
+        headers: {
+          'Content-Type': 'application/x-suggestions+json',
+        },
+      });
+    }
+
+    // Format the URL (remove trailing slashes)
+    const formattedSearxngUrl = searxngUrl.replace(/\/+$/, '');
+    const autocompleteUrl = `${formattedSearxngUrl}/autocompleter?q=${encodeURIComponent(query)}`;
+
+    // Make the request to SearXNG
+    const response = await fetch(autocompleteUrl, {
+      method: 'POST', // The example XML used POST method
+      headers: {
+        'Content-Type': 'application/x-www-form-urlencoded',
+      },
+      signal: AbortSignal.timeout(3000), // 3 second timeout
+    });
+
+    if (!response.ok) {
+      console.error(
+        `SearXNG autocompleter returned status: ${response.status}`,
+      );
+      return new NextResponse(JSON.stringify([query, []]), {
+        headers: {
+          'Content-Type': 'application/x-suggestions+json',
+        },
+      });
+    }
+
+    // Get the JSON response from SearXNG
+    const suggestions = await response.json();
+
+    // Return the suggestions in the expected format
+    return new NextResponse(JSON.stringify(suggestions), {
+      headers: {
+        'Content-Type': 'application/x-suggestions+json',
+      },
+    });
+  } catch (error) {
+    console.error('Error proxying to SearXNG autocompleter:', error);
+
+    // Return an empty suggestion list on error
+    const query = new URL(request.url).searchParams.get('q') || '';
+    return new NextResponse(JSON.stringify([query, []]), {
+      headers: {
+        'Content-Type': 'application/x-suggestions+json',
+      },
+    });
+  }
+}
--- a/src/app/api/chat/cancel/route.ts
+++ b/src/app/api/chat/cancel/route.ts
@ -0,0 +1,18 @@
+import { NextRequest } from 'next/server';
+import { cancelRequest } from '@/lib/cancel-tokens';
+
+export async function POST(req: NextRequest) {
+  const { messageId } = await req.json();
+  if (!messageId) {
+    return Response.json({ error: 'Missing messageId' }, { status: 400 });
+  }
+  const cancelled = cancelRequest(messageId);
+  if (cancelled) {
+    return Response.json({ success: true });
+  } else {
+    return Response.json(
+      { error: 'No in-progress request for this messageId' },
+      { status: 404 },
+    );
+  }
+}
--- a/src/app/api/chat/route.ts
+++ b/src/app/api/chat/route.ts
@ -1,26 +1,25 @@
-import prompts from '@/lib/prompts';
-import MetaSearchAgent from '@/lib/search/metaSearchAgent';
-import crypto from 'crypto';
-import { AIMessage, BaseMessage, HumanMessage } from '@langchain/core/messages';
-import { EventEmitter } from 'stream';
-import {
-  chatModelProviders,
-  embeddingModelProviders,
-  getAvailableChatModelProviders,
-  getAvailableEmbeddingModelProviders,
-} from '@/lib/providers';
-import db from '@/lib/db';
-import { chats, messages as messagesSchema } from '@/lib/db/schema';
-import { and, eq, gt } from 'drizzle-orm';
-import { getFileDetails } from '@/lib/utils/files';
-import { BaseChatModel } from '@langchain/core/language_models/chat_models';
-import { ChatOpenAI } from '@langchain/openai';
+import { cleanupCancelToken, registerCancelToken } from '@/lib/cancel-tokens';
 import {
  getCustomOpenaiApiKey,
  getCustomOpenaiApiUrl,
  getCustomOpenaiModelName,
 } from '@/lib/config';
+import db from '@/lib/db';
+import { chats, messages as messagesSchema } from '@/lib/db/schema';
+import {
+  getAvailableChatModelProviders,
+  getAvailableEmbeddingModelProviders,
+} from '@/lib/providers';
 import { searchHandlers } from '@/lib/search';
+import { getFileDetails } from '@/lib/utils/files';
+import { getSystemPrompts } from '@/lib/utils/prompts';
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import { AIMessage, BaseMessage, HumanMessage } from '@langchain/core/messages';
+import { ChatOllama } from '@langchain/ollama';
+import { ChatOpenAI } from '@langchain/openai';
+import crypto from 'crypto';
+import { and, eq, gt } from 'drizzle-orm';
+import { EventEmitter } from 'stream';

 export const runtime = 'nodejs';
 export const dynamic = 'force-dynamic';
@ -34,6 +33,7 @@ type Message = {
 type ChatModel = {
  provider: string;
  name: string;
+  ollamaContextWindow?: number;
 };

 type EmbeddingModel = {
@ -43,13 +43,19 @@ type EmbeddingModel = {

 type Body = {
  message: Message;
-  optimizationMode: 'speed' | 'balanced' | 'quality';
+  optimizationMode: 'speed' | 'agent';
  focusMode: string;
  history: Array<[string, string]>;
  files: Array<string>;
  chatModel: ChatModel;
  embeddingModel: EmbeddingModel;
  systemInstructions: string;
+  selectedSystemPromptIds: string[];
+};
+
+type ModelStats = {
+  modelName: string;
+  responseTime?: number;
 };

 const handleEmitterEvents = async (
@ -58,9 +64,43 @@ const handleEmitterEvents = async (
  encoder: TextEncoder,
  aiMessageId: string,
  chatId: string,
+  startTime: number,
+  userMessageId: string,
+  abortController: AbortController,
 ) => {
  let recievedMessage = '';
  let sources: any[] = [];
+  let searchQuery: string | undefined;
+  let searchUrl: string | undefined;
+  let isStreamActive = true;
+
+  // Keep-alive ping mechanism to prevent reverse proxy timeouts
+  const pingInterval = setInterval(() => {
+    if (isStreamActive) {
+      try {
+        writer.write(
+          encoder.encode(
+            JSON.stringify({
+              type: 'ping',
+              timestamp: Date.now(),
+            }) + '\n',
+          ),
+        );
+      } catch (error) {
+        // If writing fails, the connection is likely closed
+        clearInterval(pingInterval);
+        isStreamActive = false;
+      }
+    } else {
+      clearInterval(pingInterval);
+    }
+  }, 30000); // Send ping every 30 seconds
+
+  // Clean up ping interval if request is cancelled
+  abortController.signal.addEventListener('abort', () => {
+    isStreamActive = false;
+    clearInterval(pingInterval);
+  });

  stream.on('data', (data) => {
    const parsedData = JSON.parse(data);
@ -77,12 +117,22 @@ const handleEmitterEvents = async (

      recievedMessage += parsedData.data;
    } else if (parsedData.type === 'sources') {
+      // Capture the search query if available
+      if (parsedData.searchQuery) {
+        searchQuery = parsedData.searchQuery;
+      }
+      if (parsedData.searchUrl) {
+        searchUrl = parsedData.searchUrl;
+      }
+
      writer.write(
        encoder.encode(
          JSON.stringify({
            type: 'sources',
            data: parsedData.data,
+            searchQuery: parsedData.searchQuery,
            messageId: aiMessageId,
+            searchUrl: searchUrl,
          }) + '\n',
        ),
      );
@ -90,17 +140,72 @@ const handleEmitterEvents = async (
      sources = parsedData.data;
    }
  });
+
+  stream.on('agent_action', (data) => {
+    writer.write(
+      encoder.encode(
+        JSON.stringify({
+          type: 'agent_action',
+          data: data.data,
+          messageId: userMessageId,
+        }) + '\n',
+      ),
+    );
+  });
+  let modelStats: ModelStats = {
+    modelName: '',
+  };
+
+  stream.on('progress', (data) => {
+    const parsedData = JSON.parse(data);
+    if (parsedData.type === 'progress') {
+      writer.write(
+        encoder.encode(
+          JSON.stringify({
+            type: 'progress',
+            data: parsedData.data,
+            messageId: aiMessageId,
+          }) + '\n',
+        ),
+      );
+    }
+  });
+
+  stream.on('stats', (data) => {
+    const parsedData = JSON.parse(data);
+    if (parsedData.type === 'modelStats') {
+      modelStats = parsedData.data;
+    }
+  });
+
  stream.on('end', () => {
+    isStreamActive = false;
+    clearInterval(pingInterval);
+
+    const endTime = Date.now();
+    const duration = endTime - startTime;
+
+    modelStats = {
+      ...modelStats,
+      responseTime: duration,
+    };
+
    writer.write(
      encoder.encode(
        JSON.stringify({
          type: 'messageEnd',
          messageId: aiMessageId,
+          modelStats: modelStats,
+          searchQuery: searchQuery,
+          searchUrl: searchUrl,
        }) + '\n',
      ),
    );
    writer.close();

+    // Clean up the abort controller reference
+    cleanupCancelToken(userMessageId);
+
    db.insert(messagesSchema)
      .values({
        content: recievedMessage,
@ -110,11 +215,17 @@ const handleEmitterEvents = async (
        metadata: JSON.stringify({
          createdAt: new Date(),
          ...(sources && sources.length > 0 && { sources }),
+          ...(searchQuery && { searchQuery }),
+          modelStats: modelStats,
+          ...(searchUrl && { searchUrl }),
        }),
      })
      .execute();
  });
  stream.on('error', (data) => {
+    isStreamActive = false;
+    clearInterval(pingInterval);
+
    const parsedData = JSON.parse(data);
    writer.write(
      encoder.encode(
@ -169,6 +280,16 @@ const handleHistorySave = async (
      })
      .execute();
  } else {
+    await db
+      .update(messagesSchema)
+      .set({
+        content: message.content,
+        metadata: JSON.stringify({
+          createdAt: new Date(),
+        }),
+      })
+      .where(eq(messagesSchema.messageId, humanMessageId))
+      .execute();
    await db
      .delete(messagesSchema)
      .where(
@ -183,8 +304,9 @@ const handleHistorySave = async (

 export const POST = async (req: Request) => {
  try {
+    const startTime = Date.now();
    const body = (await req.json()) as Body;
-    const { message } = body;
+    const { message, selectedSystemPromptIds } = body;

    if (message.content === '') {
      return Response.json(
@ -225,13 +347,18 @@ export const POST = async (req: Request) => {
      llm = new ChatOpenAI({
        openAIApiKey: getCustomOpenaiApiKey(),
        modelName: getCustomOpenaiModelName(),
-        temperature: 0.7,
+        // temperature: 0.7,
        configuration: {
          baseURL: getCustomOpenaiApiUrl(),
        },
      }) as unknown as BaseChatModel;
    } else if (chatModelProvider && chatModel) {
      llm = chatModel.model;
+
+      // Set context window size for Ollama models
+      if (llm instanceof ChatOllama && body.chatModel?.provider === 'ollama') {
+        llm.numCtx = body.chatModel.ollamaContextWindow || 2048;
+      }
    }

    if (!llm) {
@ -272,6 +399,37 @@ export const POST = async (req: Request) => {
      );
    }

+    let systemInstructionsContent = '';
+    let personaInstructionsContent = '';
+
+    // Retrieve system prompts from database using shared utility
+    const promptData = await getSystemPrompts(selectedSystemPromptIds);
+    systemInstructionsContent = promptData.systemInstructions;
+    personaInstructionsContent = promptData.personaInstructions;
+
+    const responseStream = new TransformStream();
+    const writer = responseStream.writable.getWriter();
+    const encoder = new TextEncoder();
+
+    // --- Cancellation logic ---
+    const abortController = new AbortController();
+    registerCancelToken(message.messageId, abortController);
+
+    abortController.signal.addEventListener('abort', () => {
+      console.log('Stream aborted, sending cancel event');
+      writer.write(
+        encoder.encode(
+          JSON.stringify({
+            type: 'error',
+            data: 'Request cancelled by user',
+          }),
+        ),
+      );
+      writer.close();
+      cleanupCancelToken(message.messageId);
+    });
+
+    // Pass the abort signal to the search handler
    const stream = await handler.searchAndAnswer(
      message.content,
      history,
@ -279,14 +437,23 @@ export const POST = async (req: Request) => {
      embedding,
      body.optimizationMode,
      body.files,
-      body.systemInstructions,
+      systemInstructionsContent,
+      abortController.signal,
+      personaInstructionsContent,
+      body.focusMode,
    );

-    const responseStream = new TransformStream();
-    const writer = responseStream.writable.getWriter();
-    const encoder = new TextEncoder();
+    handleEmitterEvents(
+      stream,
+      writer,
+      encoder,
+      aiMessageId,
+      message.chatId,
+      startTime,
+      message.messageId,
+      abortController,
+    );

-    handleEmitterEvents(stream, writer, encoder, aiMessageId, message.chatId);
    handleHistorySave(message, humanMessageId, body.focusMode, body.files);

    return new Response(responseStream.readable, {
--- a/src/app/api/config/route.ts
+++ b/src/app/api/config/route.ts
@ -1,5 +1,6 @@
 import {
  getAnthropicApiKey,
+  getBaseUrl,
  getCustomOpenaiApiKey,
  getCustomOpenaiApiUrl,
  getCustomOpenaiModelName,
@ -7,9 +8,11 @@ import {
  getGroqApiKey,
  getOllamaApiEndpoint,
  getOpenaiApiKey,
+  getOpenrouterApiKey,
  getDeepseekApiKey,
  getAimlApiKey,
  getLMStudioApiEndpoint,
+  getHiddenModels,
  updateConfig,
 } from '@/lib/config';
 import {
@ -51,17 +54,28 @@ export const GET = async (req: Request) => {
      });
    }

-    config['openaiApiKey'] = getOpenaiApiKey();
+    // Helper function to obfuscate API keys
+    const protectApiKey = (key: string | null | undefined) => {
+      return key ? 'protected' : key;
+    };
+
+    // Obfuscate all API keys in the response
+    config['openaiApiKey'] = protectApiKey(getOpenaiApiKey());
+    config['groqApiKey'] = protectApiKey(getGroqApiKey());
+    config['anthropicApiKey'] = protectApiKey(getAnthropicApiKey());
+    config['geminiApiKey'] = protectApiKey(getGeminiApiKey());
+    config['deepseekApiKey'] = protectApiKey(getDeepseekApiKey());
+    config['openrouterApiKey'] = protectApiKey(getOpenrouterApiKey());
+    config['customOpenaiApiKey'] = protectApiKey(getCustomOpenaiApiKey());
+    config['aimlApiKey'] = protectApiKey(getAimlApiKey());
+
+    // Non-sensitive values remain unchanged
    config['ollamaApiUrl'] = getOllamaApiEndpoint();
    config['lmStudioApiUrl'] = getLMStudioApiEndpoint();
-    config['anthropicApiKey'] = getAnthropicApiKey();
-    config['groqApiKey'] = getGroqApiKey();
-    config['geminiApiKey'] = getGeminiApiKey();
-    config['deepseekApiKey'] = getDeepseekApiKey();
-    config['aimlApiKey'] = getAimlApiKey();
    config['customOpenaiApiUrl'] = getCustomOpenaiApiUrl();
-    config['customOpenaiApiKey'] = getCustomOpenaiApiKey();
    config['customOpenaiModelName'] = getCustomOpenaiModelName();
+    config['baseUrl'] = getBaseUrl();
+    config['hiddenModels'] = getHiddenModels();

    return Response.json({ ...config }, { status: 200 });
  } catch (err) {
@ -77,35 +91,69 @@ export const POST = async (req: Request) => {
  try {
    const config = await req.json();

+    const getUpdatedProtectedValue = (
+      newValue: string,
+      currentConfig: string,
+    ) => {
+      if (newValue === 'protected') {
+        return currentConfig;
+      }
+      return newValue;
+    };
+
    const updatedConfig = {
+      GENERAL: {
+        HIDDEN_MODELS: config.hiddenModels || [],
+      },
      MODELS: {
        OPENAI: {
-          API_KEY: config.openaiApiKey,
+          API_KEY: getUpdatedProtectedValue(
+            config.openaiApiKey,
+            getOpenaiApiKey(),
+          ),
        },
        GROQ: {
-          API_KEY: config.groqApiKey,
+          API_KEY: getUpdatedProtectedValue(config.groqApiKey, getGroqApiKey()),
        },
        ANTHROPIC: {
-          API_KEY: config.anthropicApiKey,
+          API_KEY: getUpdatedProtectedValue(
+            config.anthropicApiKey,
+            getAnthropicApiKey(),
+          ),
        },
        GEMINI: {
-          API_KEY: config.geminiApiKey,
+          API_KEY: getUpdatedProtectedValue(
+            config.geminiApiKey,
+            getGeminiApiKey(),
+          ),
        },
        OLLAMA: {
          API_URL: config.ollamaApiUrl,
        },
        DEEPSEEK: {
-          API_KEY: config.deepseekApiKey,
+          API_KEY: getUpdatedProtectedValue(
+            config.deepseekApiKey,
+            getDeepseekApiKey(),
+          ),
        },
        AIMLAPI: {
-          API_KEY: config.aimlApiKey,
+          API_KEY: getUpdatedProtectedValue(config.aimlApiKey, getAimlApiKey()),
        },
        LM_STUDIO: {
          API_URL: config.lmStudioApiUrl,
        },
+        OPENROUTER: {
+          API_KEY: getUpdatedProtectedValue(
+            config.openrouterApiKey,
+            getOpenrouterApiKey(),
+          ),
+        },
        CUSTOM_OPENAI: {
          API_URL: config.customOpenaiApiUrl,
-          API_KEY: config.customOpenaiApiKey,
+          API_KEY: getUpdatedProtectedValue(
+            config.customOpenaiApiKey,
+            getCustomOpenaiApiKey(),
+          ),
          MODEL_NAME: config.customOpenaiModelName,
        },
      },
--- a/src/app/api/images/route.ts
+++ b/src/app/api/images/route.ts
@ -5,19 +5,23 @@ import {
  getCustomOpenaiModelName,
 } from '@/lib/config';
 import { getAvailableChatModelProviders } from '@/lib/providers';
+import { getSystemInstructionsOnly } from '@/lib/utils/prompts';
 import { BaseChatModel } from '@langchain/core/language_models/chat_models';
 import { AIMessage, BaseMessage, HumanMessage } from '@langchain/core/messages';
+import { ChatOllama } from '@langchain/ollama';
 import { ChatOpenAI } from '@langchain/openai';

 interface ChatModel {
  provider: string;
  model: string;
+  ollamaContextWindow?: number;
 }

 interface ImageSearchBody {
  query: string;
  chatHistory: any[];
  chatModel?: ChatModel;
+  selectedSystemPromptIds?: string[];
 }

 export const POST = async (req: Request) => {
@ -51,25 +55,46 @@ export const POST = async (req: Request) => {
      llm = new ChatOpenAI({
        openAIApiKey: getCustomOpenaiApiKey(),
        modelName: getCustomOpenaiModelName(),
-        temperature: 0.7,
+        // temperature: 0.7,
        configuration: {
          baseURL: getCustomOpenaiApiUrl(),
        },
      }) as unknown as BaseChatModel;
    } else if (chatModelProvider && chatModel) {
      llm = chatModel.model;
+      // Set context window size for Ollama models
+      if (llm instanceof ChatOllama && body.chatModel?.provider === 'ollama') {
+        llm.numCtx = body.chatModel.ollamaContextWindow || 2048;
+      }
    }

    if (!llm) {
      return Response.json({ error: 'Invalid chat model' }, { status: 400 });
    }

+    let systemInstructions = '';
+    if (
+      body.selectedSystemPromptIds &&
+      body.selectedSystemPromptIds.length > 0
+    ) {
+      try {
+        const promptInstructions = await getSystemInstructionsOnly(
+          body.selectedSystemPromptIds,
+        );
+        systemInstructions = promptInstructions || systemInstructions;
+      } catch (error) {
+        console.error('Error fetching system prompts:', error);
+        // Continue with fallback systemInstructions
+      }
+    }
+
    const images = await handleImageSearch(
      {
        chat_history: chatHistory,
        query: body.query,
      },
      llm,
+      systemInstructions,
    );

    return Response.json({ images }, { status: 200 });
--- a/src/app/api/models/route.ts
+++ b/src/app/api/models/route.ts
@ -5,9 +5,12 @@ import {

 export const GET = async (req: Request) => {
  try {
+    const url = new URL(req.url);
+    const includeHidden = url.searchParams.get('include_hidden') === 'true';
+
    const [chatModelProviders, embeddingModelProviders] = await Promise.all([
-      getAvailableChatModelProviders(),
-      getAvailableEmbeddingModelProviders(),
+      getAvailableChatModelProviders({ includeHidden }),
+      getAvailableEmbeddingModelProviders({ includeHidden }),
    ]);

    Object.keys(chatModelProviders).forEach((provider) => {
--- a/src/app/api/opensearch/route.ts
+++ b/src/app/api/opensearch/route.ts
@ -0,0 +1,70 @@
+import { NextResponse } from 'next/server';
+import { getBaseUrl } from '@/lib/config';
+
+/**
+ * Creates an OpenSearch XML response with the given origin URL
+ */
+function generateOpenSearchResponse(origin: string): NextResponse {
+  const opensearchXml = `<?xml version="1.0" encoding="utf-8"?>
+<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/" xmlns:moz="http://www.mozilla.org/2006/browser/search/">
+  <ShortName>Perplexica</ShortName>
+  <LongName>Search with Perplexica AI</LongName>
+  <Description>Perplexica is a powerful AI-driven search engine that understands your queries and delivers relevant results.</Description>
+  <InputEncoding>UTF-8</InputEncoding>
+  <Image width="16" height="16" type="image/x-icon">${origin}/favicon.ico</Image>
+  <Url type="text/html" template="${origin}/?q={searchTerms}"/>
+  <Url rel="suggestions" type="application/x-suggestions+json" template="${origin}/api/autocomplete?q={searchTerms}"/>
+  <Url type="application/opensearchdescription+xml" rel="self" template="${origin}/api/opensearch"/>
+</OpenSearchDescription>`;
+
+  return new NextResponse(opensearchXml, {
+    headers: {
+      'Content-Type': 'application/opensearchdescription+xml',
+    },
+  });
+}
+
+export async function GET(request: Request) {
+  // Check if a BASE_URL is explicitly configured
+  const configBaseUrl = getBaseUrl();
+
+  // If BASE_URL is configured, use it, otherwise detect from request
+  if (configBaseUrl) {
+    // Remove any trailing slashes for consistency
+    let origin = configBaseUrl.replace(/\/+$/, '');
+    return generateOpenSearchResponse(origin);
+  }
+
+  // Detect the correct origin, taking into account reverse proxy headers
+  const url = new URL(request.url);
+  let origin = url.origin;
+
+  // Extract headers
+  const headers = Object.fromEntries(request.headers);
+
+  // Check for X-Forwarded-Host and related headers to handle reverse proxies
+  if (headers['x-forwarded-host']) {
+    // Determine protocol: prefer X-Forwarded-Proto, fall back to original or https
+    const protocol =
+      headers['x-forwarded-proto'] || url.protocol.replace(':', '');
+    // Build the correct public-facing origin
+    origin = `${protocol}://${headers['x-forwarded-host']}`;
+
+    // Handle non-standard ports if specified in X-Forwarded-Port
+    if (headers['x-forwarded-port']) {
+      const port = headers['x-forwarded-port'];
+      // Don't append standard ports (80 for HTTP, 443 for HTTPS)
+      if (
+        !(
+          (protocol === 'http' && port === '80') ||
+          (protocol === 'https' && port === '443')
+        )
+      ) {
+        origin = `${origin}:${port}`;
+      }
+    }
+  }
+
+  // Generate and return the OpenSearch response
+  return generateOpenSearchResponse(origin);
+}
--- a/src/app/api/search/route.ts
+++ b/src/app/api/search/route.ts
@ -13,12 +13,15 @@ import {
  getCustomOpenaiModelName,
 } from '@/lib/config';
 import { searchHandlers } from '@/lib/search';
+import { getSystemPrompts } from '@/lib/utils/prompts';
+import { ChatOllama } from '@langchain/ollama';

 interface chatModel {
  provider: string;
  name: string;
  customOpenAIKey?: string;
  customOpenAIBaseURL?: string;
+  ollamaContextWindow?: number;
 }

 interface embeddingModel {
@ -27,14 +30,14 @@ interface embeddingModel {
 }

 interface ChatRequestBody {
-  optimizationMode: 'speed' | 'balanced';
+  optimizationMode: 'speed' | 'agent';
  focusMode: string;
  chatModel?: chatModel;
  embeddingModel?: embeddingModel;
  query: string;
  history: Array<[string, string]>;
  stream?: boolean;
-  systemInstructions?: string;
+  selectedSystemPromptIds?: string[];
 }

 export const POST = async (req: Request) => {
@ -49,7 +52,7 @@ export const POST = async (req: Request) => {
    }

    body.history = body.history || [];
-    body.optimizationMode = body.optimizationMode || 'balanced';
+    body.optimizationMode = body.optimizationMode || 'speed';
    body.stream = body.stream || false;

    const history: BaseMessage[] = body.history.map((msg) => {
@ -83,7 +86,7 @@ export const POST = async (req: Request) => {
        modelName: body.chatModel?.name || getCustomOpenaiModelName(),
        openAIApiKey:
          body.chatModel?.customOpenAIKey || getCustomOpenaiApiKey(),
-        temperature: 0.7,
+        // temperature: 0.7,
        configuration: {
          baseURL:
            body.chatModel?.customOpenAIBaseURL || getCustomOpenaiApiUrl(),
@ -97,6 +100,10 @@ export const POST = async (req: Request) => {
        .model as unknown as BaseChatModel | undefined;
    }

+    if (llm instanceof ChatOllama && body.chatModel?.provider === 'ollama') {
+      llm.numCtx = body.chatModel.ollamaContextWindow || 2048;
+    }
+
    if (
      embeddingModelProviders[embeddingModelProvider] &&
      embeddingModelProviders[embeddingModelProvider][embeddingModel]
@ -118,6 +125,12 @@ export const POST = async (req: Request) => {
    if (!searchHandler) {
      return Response.json({ message: 'Invalid focus mode' }, { status: 400 });
    }
+    const abortController = new AbortController();
+    const { signal } = abortController;
+
+    const promptData = await getSystemPrompts(
+      body.selectedSystemPromptIds || [],
+    );

    const emitter = await searchHandler.searchAndAnswer(
      body.query,
@ -126,7 +139,10 @@ export const POST = async (req: Request) => {
      embeddings,
      body.optimizationMode,
      [],
-      body.systemInstructions || '',
+      promptData.systemInstructions,
+      signal,
+      promptData.personaInstructions,
+      body.focusMode,
    );

    if (!body.stream) {
@ -174,12 +190,10 @@ export const POST = async (req: Request) => {

    const encoder = new TextEncoder();

-    const abortController = new AbortController();
-    const { signal } = abortController;
-
    const stream = new ReadableStream({
      start(controller) {
        let sources: any[] = [];
+        let isStreamActive = true;

        controller.enqueue(
          encoder.encode(
@ -190,7 +204,31 @@ export const POST = async (req: Request) => {
          ),
        );

+        // Keep-alive ping mechanism to prevent reverse proxy timeouts
+        const pingInterval = setInterval(() => {
+          if (isStreamActive && !signal.aborted) {
+            try {
+              controller.enqueue(
+                encoder.encode(
+                  JSON.stringify({
+                    type: 'ping',
+                    timestamp: Date.now(),
+                  }) + '\n',
+                ),
+              );
+            } catch (error) {
+              // If enqueueing fails, the connection is likely closed
+              clearInterval(pingInterval);
+              isStreamActive = false;
+            }
+          } else {
+            clearInterval(pingInterval);
+          }
+        }, 30000); // Send ping every 30 seconds
+
        signal.addEventListener('abort', () => {
+          isStreamActive = false;
+          clearInterval(pingInterval);
          emitter.removeAllListeners();

          try {
@ -232,6 +270,9 @@ export const POST = async (req: Request) => {
        emitter.on('end', () => {
          if (signal.aborted) return;

+          isStreamActive = false;
+          clearInterval(pingInterval);
+
          controller.enqueue(
            encoder.encode(
              JSON.stringify({
@ -245,6 +286,9 @@ export const POST = async (req: Request) => {
        emitter.on('error', (error: any) => {
          if (signal.aborted) return;

+          isStreamActive = false;
+          clearInterval(pingInterval);
+
          controller.error(error);
        });
      },
--- a/src/app/api/suggestions/route.ts
+++ b/src/app/api/suggestions/route.ts
@ -5,18 +5,22 @@ import {
  getCustomOpenaiModelName,
 } from '@/lib/config';
 import { getAvailableChatModelProviders } from '@/lib/providers';
+import { getSystemInstructionsOnly } from '@/lib/utils/prompts';
 import { BaseChatModel } from '@langchain/core/language_models/chat_models';
 import { AIMessage, BaseMessage, HumanMessage } from '@langchain/core/messages';
 import { ChatOpenAI } from '@langchain/openai';
+import { ChatOllama } from '@langchain/ollama';

 interface ChatModel {
  provider: string;
  model: string;
+  ollamaContextWindow?: number;
 }

 interface SuggestionsGenerationBody {
  chatHistory: any[];
  chatModel?: ChatModel;
+  selectedSystemPromptIds?: string[];
 }

 export const POST = async (req: Request) => {
@ -50,24 +54,45 @@ export const POST = async (req: Request) => {
      llm = new ChatOpenAI({
        openAIApiKey: getCustomOpenaiApiKey(),
        modelName: getCustomOpenaiModelName(),
-        temperature: 0.7,
+        // temperature: 0.7,
        configuration: {
          baseURL: getCustomOpenaiApiUrl(),
        },
      }) as unknown as BaseChatModel;
    } else if (chatModelProvider && chatModel) {
      llm = chatModel.model;
+      // Set context window size for Ollama models
+      if (llm instanceof ChatOllama && body.chatModel?.provider === 'ollama') {
+        llm.numCtx = body.chatModel.ollamaContextWindow || 2048;
+      }
    }

    if (!llm) {
      return Response.json({ error: 'Invalid chat model' }, { status: 400 });
    }

+    let systemInstructions = '';
+    if (
+      body.selectedSystemPromptIds &&
+      body.selectedSystemPromptIds.length > 0
+    ) {
+      try {
+        const retrievedInstructions = await getSystemInstructionsOnly(
+          body.selectedSystemPromptIds,
+        );
+        systemInstructions = retrievedInstructions;
+      } catch (error) {
+        console.error('Error retrieving system prompts:', error);
+        // Continue with existing systemInstructions as fallback
+      }
+    }
+
    const suggestions = await generateSuggestions(
      {
        chat_history: chatHistory,
      },
      llm,
+      systemInstructions,
    );

    return Response.json({ suggestions }, { status: 200 });
--- a/src/app/api/system-prompts/[id]/route.ts
+++ b/src/app/api/system-prompts/[id]/route.ts
@ -0,0 +1,76 @@
+import db from '@/lib/db';
+import { systemPrompts } from '@/lib/db/schema';
+import { eq } from 'drizzle-orm';
+import { NextResponse } from 'next/server';
+
+export async function PUT(
+  req: Request,
+  { params }: { params: Promise<{ id: string }> },
+) {
+  try {
+    const { id } = await params;
+    const { name, content, type } = await req.json();
+    if (!name || !content) {
+      return NextResponse.json(
+        { error: 'Name and content are required' },
+        { status: 400 },
+      );
+    }
+    if (type && type !== 'system' && type !== 'persona') {
+      return NextResponse.json(
+        { error: 'Type must be either "system" or "persona"' },
+        { status: 400 },
+      );
+    }
+
+    const updateData: any = { name, content, updatedAt: new Date() };
+    if (type) {
+      updateData.type = type;
+    }
+
+    const updatedPrompt = await db
+      .update(systemPrompts)
+      .set(updateData)
+      .where(eq(systemPrompts.id, id))
+      .returning();
+    if (updatedPrompt.length === 0) {
+      return NextResponse.json(
+        { error: 'System prompt not found' },
+        { status: 404 },
+      );
+    }
+    return NextResponse.json(updatedPrompt[0]);
+  } catch (error) {
+    console.error('Failed to update system prompt:', error);
+    return NextResponse.json(
+      { error: 'Failed to update system prompt' },
+      { status: 500 },
+    );
+  }
+}
+
+export async function DELETE(
+  req: Request,
+  { params }: { params: Promise<{ id: string }> },
+) {
+  try {
+    const { id } = await params;
+    const deletedPrompt = await db
+      .delete(systemPrompts)
+      .where(eq(systemPrompts.id, id))
+      .returning();
+    if (deletedPrompt.length === 0) {
+      return NextResponse.json(
+        { error: 'System prompt not found' },
+        { status: 404 },
+      );
+    }
+    return NextResponse.json({ message: 'System prompt deleted successfully' });
+  } catch (error) {
+    console.error('Failed to delete system prompt:', error);
+    return NextResponse.json(
+      { error: 'Failed to delete system prompt' },
+      { status: 500 },
+    );
+  }
+}
--- a/src/app/api/system-prompts/route.ts
+++ b/src/app/api/system-prompts/route.ts
@ -0,0 +1,69 @@
+import db from '@/lib/db';
+import { systemPrompts } from '@/lib/db/schema';
+import { NextResponse } from 'next/server';
+import { asc, eq } from 'drizzle-orm';
+
+export async function GET(req: Request) {
+  try {
+    const { searchParams } = new URL(req.url);
+    const type = searchParams.get('type');
+
+    let prompts;
+
+    if (type && (type === 'system' || type === 'persona')) {
+      prompts = await db
+        .select()
+        .from(systemPrompts)
+        .where(eq(systemPrompts.type, type))
+        .orderBy(asc(systemPrompts.name));
+    } else {
+      prompts = await db
+        .select()
+        .from(systemPrompts)
+        .orderBy(asc(systemPrompts.name));
+    }
+
+    return NextResponse.json(prompts);
+  } catch (error) {
+    console.error('Failed to fetch system prompts:', error);
+    return NextResponse.json(
+      { error: 'Failed to fetch system prompts' },
+      { status: 500 },
+    );
+  }
+}
+
+export async function POST(req: Request) {
+  try {
+    const { name, content, type = 'system' } = await req.json();
+    if (!name || !content) {
+      return NextResponse.json(
+        { error: 'Name and content are required' },
+        { status: 400 },
+      );
+    }
+    if (type && type !== 'system' && type !== 'persona') {
+      return NextResponse.json(
+        { error: 'Type must be either "system" or "persona"' },
+        { status: 400 },
+      );
+    }
+    const newPrompt = await db
+      .insert(systemPrompts)
+      .values({
+        name,
+        content,
+        type,
+        createdAt: new Date(),
+        updatedAt: new Date(),
+      })
+      .returning();
+    return NextResponse.json(newPrompt[0], { status: 201 });
+  } catch (error) {
+    console.error('Failed to create system prompt:', error);
+    return NextResponse.json(
+      { error: 'Failed to create system prompt' },
+      { status: 500 },
+    );
+  }
+}
--- a/src/app/api/uploads/route.ts
+++ b/src/app/api/uploads/route.ts
@ -2,11 +2,21 @@ import { NextResponse } from 'next/server';
 import fs from 'fs';
 import path from 'path';
 import crypto from 'crypto';
-import { getAvailableEmbeddingModelProviders } from '@/lib/providers';
+import { getAvailableEmbeddingModelProviders, getAvailableChatModelProviders } from '@/lib/providers';
+import { 
+  getCustomOpenaiApiKey,
+  getCustomOpenaiApiUrl,
+  getCustomOpenaiModelName,
+} from '@/lib/config';
 import { PDFLoader } from '@langchain/community/document_loaders/fs/pdf';
 import { DocxLoader } from '@langchain/community/document_loaders/fs/docx';
 import { RecursiveCharacterTextSplitter } from '@langchain/textsplitters';
 import { Document } from 'langchain/document';
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import { ChatOpenAI } from '@langchain/openai';
+import { ChatOllama } from '@langchain/ollama';
+import { z } from 'zod';
+import { withStructuredOutput } from '@/lib/utils/structuredOutput';

 interface FileRes {
  fileName: string;
@ -25,6 +35,52 @@ const splitter = new RecursiveCharacterTextSplitter({
  chunkOverlap: 100,
 });

+// Define Zod schema for structured topic generation output
+const TopicsSchema = z.object({
+  topics: z
+    .array(z.string())
+    .min(1)
+    .max(3)
+    .describe('Array of 1-3 concise, descriptive topics that capture the main subject matter'),
+});
+
+type TopicsOutput = z.infer<typeof TopicsSchema>;
+
+/**
+ * Generate semantic topics for a document using LLM with structured output
+ */
+async function generateFileTopics(
+  content: string,
+  filename: string,
+  llm: BaseChatModel
+): Promise<string> {
+  try {
+    // Take first 1500 characters for topic generation to avoid token limits
+    const excerpt = content.substring(0, 1500);
+    
+    const prompt = `Analyze the following document excerpt and generate 1-5 concise, descriptive topics that capture the main subject matter. The topics should be useful for determining if this document is relevant to answer questions.
+
+Document filename: ${filename}
+Document excerpt:
+${excerpt}
+
+Generate topics that describe what this document is about, its domain, and key subject areas. Focus on topics that would help determine relevance for search queries.`;
+
+    // Use structured output for reliable topic extraction
+    const structuredLlm = withStructuredOutput(llm, TopicsSchema, {
+      name: 'generate_topics',
+    });
+
+    const result = await structuredLlm.invoke(prompt);
+    console.log('Generated topics:', result.topics);
+    // Filename is included for context
+    return filename + ', ' + result.topics.join(', ');
+  } catch (error) {
+    console.warn('Error generating topics with LLM:', error);
+    return `Document: ${filename}`;
+  }
+}
+
 export async function POST(req: Request) {
  try {
    const formData = await req.formData();
@ -32,6 +88,9 @@ export async function POST(req: Request) {
    const files = formData.getAll('files') as File[];
    const embedding_model = formData.get('embedding_model');
    const embedding_model_provider = formData.get('embedding_model_provider');
+    const chat_model = formData.get('chat_model');
+    const chat_model_provider = formData.get('chat_model_provider');
+    const ollama_context_window = formData.get('ollama_context_window');

    if (!embedding_model || !embedding_model_provider) {
      return NextResponse.json(
@ -40,21 +99,65 @@ export async function POST(req: Request) {
      );
    }

-    const embeddingModels = await getAvailableEmbeddingModelProviders();
-    const provider =
-      embedding_model_provider ?? Object.keys(embeddingModels)[0];
-    const embeddingModel =
-      embedding_model ?? Object.keys(embeddingModels[provider as string])[0];
+    // Get available providers
+    const [chatModelProviders, embeddingModelProviders] = await Promise.all([
+      getAvailableChatModelProviders(),
+      getAvailableEmbeddingModelProviders(),
+    ]);

-    let embeddingsModel =
-      embeddingModels[provider as string]?.[embeddingModel as string]?.model;
-    if (!embeddingsModel) {
+    // Setup embedding model
+    const embeddingProvider =
+      embeddingModelProviders[
+        embedding_model_provider as string ?? Object.keys(embeddingModelProviders)[0]
+      ];
+    const embeddingModelConfig =
+      embeddingProvider[
+        embedding_model as string ?? Object.keys(embeddingProvider)[0]
+      ];
+
+    if (!embeddingModelConfig) {
      return NextResponse.json(
        { message: 'Invalid embedding model selected' },
        { status: 400 },
      );
    }

+    let embeddingsModel = embeddingModelConfig.model;
+
+    // Setup chat model for topic generation (similar to chat route)
+    const chatModelProvider =
+      chatModelProviders[
+        chat_model_provider as string ?? Object.keys(chatModelProviders)[0]
+      ];
+    const chatModelConfig =
+      chatModelProvider[
+        chat_model as string ?? Object.keys(chatModelProvider)[0]
+      ];
+
+    let llm: BaseChatModel;
+
+    // Handle chat model creation like in chat route
+    if (chat_model_provider === 'custom_openai') {
+      llm = new ChatOpenAI({
+        openAIApiKey: getCustomOpenaiApiKey(),
+        modelName: getCustomOpenaiModelName(),
+        temperature: 0.1,
+        configuration: {
+          baseURL: getCustomOpenaiApiUrl(),
+        },
+      }) as unknown as BaseChatModel;
+    } else if (chatModelProvider && chatModelConfig) {
+      llm = chatModelConfig.model;
+
+      // Set context window size for Ollama models
+      if (llm instanceof ChatOllama && chat_model_provider === 'ollama') {
+        // Use provided context window or default to 2048
+        const contextWindow = ollama_context_window ? 
+          parseInt(ollama_context_window as string, 10) : 2048;
+        llm.numCtx = contextWindow;
+      }
+    }
+
    const processedFiles: FileRes[] = [];

    await Promise.all(
@ -89,11 +192,16 @@ export async function POST(req: Request) {

        const splitted = await splitter.splitDocuments(docs);

+        // Generate semantic topics using LLM
+        const fullContent = docs.map(doc => doc.pageContent).join('\n');
+        const semanticTopics = await generateFileTopics(fullContent, file.name, llm);
+
        const extractedDataPath = filePath.replace(/\.\w+$/, '-extracted.json');
        fs.writeFileSync(
          extractedDataPath,
          JSON.stringify({
            title: file.name,
+            topics: semanticTopics,
            contents: splitted.map((doc) => doc.pageContent),
          }),
        );
--- a/src/app/api/videos/route.ts
+++ b/src/app/api/videos/route.ts
@ -5,19 +5,23 @@ import {
  getCustomOpenaiModelName,
 } from '@/lib/config';
 import { getAvailableChatModelProviders } from '@/lib/providers';
+import { getSystemInstructionsOnly } from '@/lib/utils/prompts';
 import { BaseChatModel } from '@langchain/core/language_models/chat_models';
 import { AIMessage, BaseMessage, HumanMessage } from '@langchain/core/messages';
+import { ChatOllama } from '@langchain/ollama';
 import { ChatOpenAI } from '@langchain/openai';

 interface ChatModel {
  provider: string;
  model: string;
+  ollamaContextWindow?: number;
 }

 interface VideoSearchBody {
  query: string;
  chatHistory: any[];
  chatModel?: ChatModel;
+  selectedSystemPromptIds?: string[];
 }

 export const POST = async (req: Request) => {
@ -58,18 +62,39 @@ export const POST = async (req: Request) => {
      }) as unknown as BaseChatModel;
    } else if (chatModelProvider && chatModel) {
      llm = chatModel.model;
+      // Set context window size for Ollama models
+      if (llm instanceof ChatOllama && body.chatModel?.provider === 'ollama') {
+        llm.numCtx = body.chatModel.ollamaContextWindow || 2048;
+      }
    }

    if (!llm) {
      return Response.json({ error: 'Invalid chat model' }, { status: 400 });
    }

+    let systemInstructions = '';
+    if (
+      body.selectedSystemPromptIds &&
+      body.selectedSystemPromptIds.length > 0
+    ) {
+      try {
+        const retrievedInstructions = await getSystemInstructionsOnly(
+          body.selectedSystemPromptIds,
+        );
+        systemInstructions = retrievedInstructions;
+      } catch (error) {
+        console.error('Error retrieving system prompts:', error);
+        // Continue with existing systemInstructions as fallback
+      }
+    }
+
    const videos = await handleVideoSearch(
      {
        chat_history: chatHistory,
        query: body.query,
      },
      llm,
+      systemInstructions,
    );

    return Response.json({ videos }, { status: 200 });
--- a/src/app/globals.css
+++ b/src/app/globals.css
@ -12,6 +12,20 @@
  }
 }

+@layer utilities {
+  @keyframes high-bounce {
+    0%,
+    100% {
+      transform: translateY(0);
+      animation-timing-function: cubic-bezier(0.8, 0, 1, 1);
+    }
+    50% {
+      transform: translateY(-9px);
+      animation-timing-function: cubic-bezier(0, 0, 0.2, 1);
+    }
+  }
+}
+
@media screen and (-webkit-min-device-pixel-ratio: 0) {
  select,
  textarea,
--- a/src/app/layout.tsx
+++ b/src/app/layout.tsx
@ -26,6 +26,14 @@ export default function RootLayout({
 }>) {
  return (
    <html className="h-full" lang="en" suppressHydrationWarning>
+      <head>
+        <link
+          rel="search"
+          type="application/opensearchdescription+xml"
+          title="Perplexica Search"
+          href="/api/opensearch"
+        />
+      </head>
      <body className={cn('h-full', montserrat.className)}>
        <ThemeProvider>
          <Sidebar>{children}</Sidebar>
--- a/src/app/settings/page.tsx
+++ b/src/app/settings/page.tsx
--- a/src/components/AgentActionDisplay.tsx
+++ b/src/components/AgentActionDisplay.tsx
@ -0,0 +1,263 @@
+'use client';
+
+import { useState } from 'react';
+import { cn } from '@/lib/utils';
+import {
+  ChevronDown,
+  ChevronUp,
+  Bot,
+  Search,
+  Zap,
+  Microscope,
+  Ban,
+  CircleCheck,
+  ListPlus,
+} from 'lucide-react';
+import { AgentActionEvent } from './ChatWindow';
+
+interface AgentActionDisplayProps {
+  events: AgentActionEvent[];
+  messageId: string;
+  isLoading: boolean;
+}
+
+const AgentActionDisplay = ({
+  events,
+  messageId,
+  isLoading,
+}: AgentActionDisplayProps) => {
+  const [isExpanded, setIsExpanded] = useState(false);
+
+  // Get the most recent event for collapsed view
+  const latestEvent = events[events.length - 1];
+
+  // Common function to format action names
+  const formatActionName = (action: string) => {
+    return action.replace(/_/g, ' ').toLocaleLowerCase();
+  };
+
+  // Function to get appropriate icon based on action type
+  const getActionIcon = (action: string, size: number = 20) => {
+    switch (action) {
+      case 'ANALYZING_PREVIEW_CONTENT':
+        return <Search size={size} className="text-[#9C27B0]" />;
+      case 'PROCESSING_PREVIEW_CONTENT':
+        return <Zap size={size} className="text-[#9C27B0]" />;
+      case 'PROCEEDING_WITH_FULL_ANALYSIS':
+        return <Microscope size={size} className="text-[#9C27B0]" />;
+      case 'SKIPPING_IRRELEVANT_SOURCE':
+        return <Ban size={size} className="text-red-600 dark:text-red-500" />;
+      case 'CONTEXT_UPDATED':
+        return (
+          <ListPlus
+            size={size}
+            className="text-green-600 dark:text-green-500"
+          />
+        );
+      case 'INFORMATION_GATHERING_COMPLETE':
+        return (
+          <CircleCheck
+            size={size}
+            className="text-green-600 dark:text-green-500"
+          />
+        );
+      default:
+        return <Bot size={size} className="text-[#9C27B0]" />;
+    }
+  };
+
+  if (!latestEvent) {
+    return null;
+  }
+
+  return (
+    <div className="my-4 bg-light-secondary/50 dark:bg-dark-secondary/50 rounded-xl border border-light-200 dark:border-dark-200 overflow-hidden">
+      <button
+        onClick={() => setIsExpanded(!isExpanded)}
+        className="w-full flex items-center justify-between px-4 py-3 text-black/90 dark:text-white/90 hover:bg-light-200 dark:hover:bg-dark-200 transition duration-200"
+      >
+        <div className="flex items-center space-x-2">
+          {getActionIcon(latestEvent.action)}
+          <span className="font-medium text-base text-black/70 dark:text-white/70 tracking-wide capitalize flex items-center">
+            {!isLoading ||
+            latestEvent.action === 'INFORMATION_GATHERING_COMPLETE'
+              ? 'Agent Log'
+              : formatActionName(latestEvent.action)}
+            {/* {isLoading &&
+              latestEvent.action !== 'INFORMATION_GATHERING_COMPLETE' && (
+                <span className="ml-2 inline-block align-middle">
+                  <span className="animate-spin inline-block w-4 h-4 border-2 border-t-transparent border-[#9C27B0] rounded-full align-middle"></span>
+                </span>
+              )} */}
+          </span>
+        </div>
+        {isExpanded ? (
+          <ChevronUp size={18} className="text-black/70 dark:text-white/70" />
+        ) : (
+          <ChevronDown size={18} className="text-black/70 dark:text-white/70" />
+        )}
+      </button>
+
+      {isExpanded && (
+        <div className="px-4 py-3 text-black/80 dark:text-white/80 text-base border-t border-light-200 dark:border-dark-200 bg-light-100/50 dark:bg-dark-100/50">
+          <div className="space-y-3">
+            {events.map((event, index) => (
+              <div
+                key={`${messageId}-${index}-${event.action}`}
+                className="flex flex-col space-y-1 p-3 bg-white/50 dark:bg-black/20 rounded-lg border border-light-200/50 dark:border-dark-200/50"
+              >
+                <div className="flex items-center space-x-2">
+                  {getActionIcon(event.action, 16)}
+                  <span className="font-medium text-sm text-black/70 dark:text-white/70 capitalize tracking-wide">
+                    {formatActionName(event.action)}
+                  </span>
+                </div>
+
+                {event.message && event.message.length > 0 && (
+                  <p className="text-base">{event.message}</p>
+                )}
+
+                {/* Display relevant details based on event type */}
+                {event.details && Object.keys(event.details).length > 0 && (
+                  <div className="mt-2 text-sm text-black/60 dark:text-white/60">
+                    {event.details.sourceUrl && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Source:
+                        </span>
+                        <span className="truncate">
+                          <a href={event.details.sourceUrl} target="_blank">
+                            {event.details.sourceUrl}
+                          </a>
+                        </span>
+                      </div>
+                    )}
+                    {event.details.skipReason && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Reason:
+                        </span>
+                        <span>{event.details.skipReason}</span>
+                      </div>
+                    )}
+                    {event.details.searchQuery &&
+                      event.details.searchQuery !== event.details.query && (
+                        <div className="flex space-x-1">
+                          <span className="font-bold whitespace-nowrap">
+                            Search Query:
+                          </span>
+                          <span className="italic">
+                            &quot;{event.details.searchQuery}&quot;
+                          </span>
+                        </div>
+                      )}
+                    {event.details.sourcesFound !== undefined && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Sources Found:
+                        </span>
+                        <span>{event.details.sourcesFound}</span>
+                      </div>
+                    )}
+                    {/* {(event.details.documentCount !== undefined && event.details.documentCount > 0) && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">Documents:</span>
+                        <span>{event.details.documentCount}</span>
+                      </div>
+                    )} */}
+                    {event.details.contentLength !== undefined && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Content Length:
+                        </span>
+                        <span>{event.details.contentLength} characters</span>
+                      </div>
+                    )}
+                    {event.details.searchInstructions !== undefined && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Search Instructions:
+                        </span>
+                        <span>{event.details.searchInstructions}</span>
+                      </div>
+                    )}
+                    {/* {event.details.previewCount !== undefined && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">Preview Sources:</span>
+                        <span>{event.details.previewCount}</span>
+                      </div>
+                    )} */}
+                    {event.details.processingType && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Processing Type:
+                        </span>
+                        <span className="capitalize">
+                          {event.details.processingType.replace('-', ' ')}
+                        </span>
+                      </div>
+                    )}
+                    {event.details.insufficiencyReason && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Reason:
+                        </span>
+                        <span>{event.details.insufficiencyReason}</span>
+                      </div>
+                    )}
+                    {event.details.reason && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Reason:
+                        </span>
+                        <span>{event.details.reason}</span>
+                      </div>
+                    )}
+                    {/* {event.details.taskCount !== undefined && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">Tasks:</span>
+                        <span>{event.details.taskCount}</span>
+                      </div>
+                    )} */}
+                    {event.details.currentTask && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Current Task:
+                        </span>
+                        <span className="italic">
+                          &quot;{event.details.currentTask}&quot;
+                        </span>
+                      </div>
+                    )}
+                    {event.details.nextTask && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Next:
+                        </span>
+                        <span className="italic">
+                          &quot;{event.details.nextTask}&quot;
+                        </span>
+                      </div>
+                    )}
+                    {event.details.currentSearchFocus && (
+                      <div className="flex space-x-1">
+                        <span className="font-bold whitespace-nowrap">
+                          Search Focus:
+                        </span>
+                        <span className="italic">
+                          &quot;{event.details.currentSearchFocus}&quot;
+                        </span>
+                      </div>
+                    )}
+                  </div>
+                )}
+              </div>
+            ))}
+          </div>
+        </div>
+      )}
+    </div>
+  );
+};
+
+export default AgentActionDisplay;
--- a/src/components/Chat.tsx
+++ b/src/components/Chat.tsx
@ -1,52 +1,132 @@
 'use client';

 import { Fragment, useEffect, useRef, useState } from 'react';
-import MessageInput from './MessageInput';
 import { File, Message } from './ChatWindow';
 import MessageBox from './MessageBox';
 import MessageBoxLoading from './MessageBoxLoading';
+import MessageInput from './MessageInput';
+import AgentActionDisplay from './AgentActionDisplay';

 const Chat = ({
  loading,
  messages,
  sendMessage,
-  messageAppeared,
+  scrollTrigger,
  rewrite,
  fileIds,
  setFileIds,
  files,
  setFiles,
+  optimizationMode,
+  setOptimizationMode,
+  focusMode,
+  setFocusMode,
+  handleEditMessage,
+  analysisProgress,
+  systemPromptIds,
+  setSystemPromptIds,
 }: {
  messages: Message[];
-  sendMessage: (message: string) => void;
+  sendMessage: (
+    message: string,
+    options?: {
+      messageId?: string;
+      rewriteIndex?: number;
+      suggestions?: string[];
+    },
+  ) => void;
  loading: boolean;
-  messageAppeared: boolean;
+  scrollTrigger: number;
  rewrite: (messageId: string) => void;
  fileIds: string[];
  setFileIds: (fileIds: string[]) => void;
  files: File[];
  setFiles: (files: File[]) => void;
+  optimizationMode: string;
+  setOptimizationMode: (mode: string) => void;
+  focusMode: string;
+  setFocusMode: (mode: string) => void;
+  handleEditMessage: (messageId: string, content: string) => void;
+  analysisProgress: {
+    message: string;
+    current: number;
+    total: number;
+  } | null;
+  systemPromptIds: string[];
+  setSystemPromptIds: (ids: string[]) => void;
 }) => {
-  const [dividerWidth, setDividerWidth] = useState(0);
-  const dividerRef = useRef<HTMLDivElement | null>(null);
+  const [isAtBottom, setIsAtBottom] = useState(true);
+  const [manuallyScrolledUp, setManuallyScrolledUp] = useState(false);
+  const [inputStyle, setInputStyle] = useState<React.CSSProperties>({});
  const messageEnd = useRef<HTMLDivElement | null>(null);
+  const containerRef = useRef<HTMLDivElement | null>(null);
+  const SCROLL_THRESHOLD = 250; // pixels from bottom to consider "at bottom"
+  const [currentMessageId, setCurrentMessageId] = useState<string | undefined>(
+    undefined,
+  );

+  // Check if user is at bottom of page
  useEffect(() => {
-    const updateDividerWidth = () => {
-      if (dividerRef.current) {
-        setDividerWidth(dividerRef.current.scrollWidth);
+    const checkIsAtBottom = () => {
+      const position = window.innerHeight + window.scrollY;
+      const height = document.body.scrollHeight;
+      const atBottom = position >= height - SCROLL_THRESHOLD;
+
+      setIsAtBottom(atBottom);
+    };
+
+    // Initial check
+    checkIsAtBottom();
+
+    // Add scroll event listener
+    window.addEventListener('scroll', checkIsAtBottom);
+
+    return () => {
+      window.removeEventListener('scroll', checkIsAtBottom);
+    };
+  }, []);
+
+  // Detect wheel and touch events to identify user's scrolling direction
+  useEffect(() => {
+    const checkIsAtBottom = () => {
+      const position = window.innerHeight + window.scrollY;
+      const height = document.body.scrollHeight;
+      const atBottom = position >= height - SCROLL_THRESHOLD;
+
+      // If user scrolls to bottom, reset the manuallyScrolledUp flag
+      if (atBottom) {
+        setManuallyScrolledUp(false);
+      }
+
+      setIsAtBottom(atBottom);
+    };
+
+    const handleWheel = (e: WheelEvent) => {
+      // Positive deltaY means scrolling down, negative means scrolling up
+      if (e.deltaY < 0) {
+        // User is scrolling up
+        setManuallyScrolledUp(true);
+      } else if (e.deltaY > 0) {
+        checkIsAtBottom();
      }
    };

-    updateDividerWidth();
+    const handleTouchStart = (e: TouchEvent) => {
+      // Immediately stop auto-scrolling on any touch interaction
+      setManuallyScrolledUp(true);
+    };

-    window.addEventListener('resize', updateDividerWidth);
+    // Add event listeners
+    window.addEventListener('wheel', handleWheel, { passive: true });
+    window.addEventListener('touchstart', handleTouchStart, { passive: true });

    return () => {
-      window.removeEventListener('resize', updateDividerWidth);
+      window.removeEventListener('wheel', handleWheel);
+      window.removeEventListener('touchstart', handleTouchStart);
    };
-  });
+  }, [isAtBottom]);

+  // Scroll when user sends a message
  useEffect(() => {
    const scroll = () => {
      messageEnd.current?.scrollIntoView({ behavior: 'smooth' });
@ -56,13 +136,79 @@ const Chat = ({
      document.title = `${messages[0].content.substring(0, 30)} - Perplexica`;
    }

-    if (messages[messages.length - 1]?.role == 'user') {
+    // Always scroll when user sends a message
+    if (messages[messages.length - 1]?.role === 'user') {
      scroll();
+      setIsAtBottom(true); // Reset to true when user sends a message
+      setManuallyScrolledUp(false); // Reset manually scrolled flag when user sends a message
    }
  }, [messages]);

+  // Auto-scroll for assistant responses only if user is at bottom and hasn't manually scrolled up
+  useEffect(() => {
+    const position = window.innerHeight + window.scrollY;
+    const height = document.body.scrollHeight;
+    const atBottom = position >= height - SCROLL_THRESHOLD;
+    setIsAtBottom(atBottom);
+
+    if (isAtBottom && !manuallyScrolledUp && messages.length > 0) {
+      messageEnd.current?.scrollIntoView({ behavior: 'smooth' });
+    }
+  }, [scrollTrigger, isAtBottom, messages.length, manuallyScrolledUp]);
+
+  // Sync input width with main container width
+  useEffect(() => {
+    const updateInputStyle = () => {
+      if (containerRef.current) {
+        const rect = containerRef.current.getBoundingClientRect();
+        setInputStyle({
+          width: rect.width,
+          left: rect.left,
+          right: window.innerWidth - rect.right,
+        });
+      }
+    };
+
+    // Initial calculation
+    updateInputStyle();
+
+    // Update on resize
+    window.addEventListener('resize', updateInputStyle);
+
+    return () => {
+      window.removeEventListener('resize', updateInputStyle);
+    };
+  }, []);
+
+  // Track the last user messageId when loading starts
+  useEffect(() => {
+    if (loading) {
+      // Find the last user message
+      const lastUserMsg = [...messages]
+        .reverse()
+        .find((m) => m.role === 'user');
+      setCurrentMessageId(lastUserMsg?.messageId);
+    } else {
+      setCurrentMessageId(undefined);
+    }
+  }, [loading, messages]);
+
+  // Cancel handler
+  const handleCancel = async () => {
+    if (!currentMessageId) return;
+    try {
+      await fetch('/api/chat/cancel', {
+        method: 'POST',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({ messageId: currentMessageId }),
+      });
+    } catch (e) {
+      // Optionally handle error
+    }
+  };
+
  return (
-    <div className="flex flex-col space-y-6 pt-8 pb-44 lg:pb-32 sm:mx-4 md:mx-8">
+    <div ref={containerRef} className="space-y-6 pt-8 pb-48 sm:mx-4 md:mx-8">
      {messages.map((msg, i) => {
        const isLast = i === messages.length - 1;

@ -74,34 +220,90 @@ const Chat = ({
              messageIndex={i}
              history={messages}
              loading={loading}
-              dividerRef={isLast ? dividerRef : undefined}
              isLast={isLast}
              rewrite={rewrite}
              sendMessage={sendMessage}
+              handleEditMessage={handleEditMessage}
            />
+            {/* Show agent actions after user messages - either completed or in progress */}
+            {msg.role === 'user' && (
+              <>
+                {/* Show agent actions if they exist */}
+                {msg.agentActions && msg.agentActions.length > 0 && (
+                  <AgentActionDisplay
+                    messageId={msg.messageId}
+                    events={msg.agentActions}
+                    isLoading={loading}
+                  />
+                )}
+                {/* Show empty agent action display if this is the last user message and we're loading */}
+                {loading &&
+                  isLast &&
+                  (!msg.agentActions || msg.agentActions.length === 0) && (
+                    <AgentActionDisplay
+                      messageId={msg.messageId}
+                      events={[]}
+                      isLoading={loading}
+                    />
+                  )}
+              </>
+            )}
            {!isLast && msg.role === 'assistant' && (
              <div className="h-px w-full bg-light-secondary dark:bg-dark-secondary" />
            )}
          </Fragment>
        );
      })}
-      {loading && !messageAppeared && <MessageBoxLoading />}
-      <div ref={messageEnd} className="h-0" />
-      {dividerWidth > 0 && (
-        <div
-          className="bottom-24 lg:bottom-10 fixed z-40"
-          style={{ width: dividerWidth }}
+      {loading && <MessageBoxLoading progress={analysisProgress} />}
+      <div className="fixed bottom-24 lg:bottom-10 z-40" style={inputStyle}>
+        {/* Scroll to bottom button - appears above the MessageInput when user has scrolled up */}
+        {manuallyScrolledUp && !isAtBottom && (
+          <div className="absolute -top-14 right-2 z-10">
+            <button
+              onClick={() => {
+                setManuallyScrolledUp(false);
+                setIsAtBottom(true);
+                messageEnd.current?.scrollIntoView({ behavior: 'smooth' });
+              }}
+              className="bg-[#24A0ED] text-white hover:bg-opacity-85 transition duration-100 rounded-full px-4 py-2 shadow-lg flex items-center justify-center"
+              aria-label="Scroll to bottom"
            >
+              <svg
+                xmlns="http://www.w3.org/2000/svg"
+                className="h-5 w-5 mr-1"
+                viewBox="0 0 20 20"
+                fill="currentColor"
+              >
+                <path
+                  fillRule="evenodd"
+                  d="M14.707 12.707a1 1 0 01-1.414 0L10 9.414l-3.293 3.293a1 1 0 01-1.414-1.414l4-4a1 1 0 011.414 0l4 4a1 1 0 010 1.414z"
+                  clipRule="evenodd"
+                  transform="rotate(180 10 10)"
+                />
+              </svg>
+              <span className="text-sm">Scroll to bottom</span>
+            </button>
+          </div>
+        )}
+
        <MessageInput
+          firstMessage={messages.length === 0}
          loading={loading}
          sendMessage={sendMessage}
          fileIds={fileIds}
          setFileIds={setFileIds}
          files={files}
          setFiles={setFiles}
+          optimizationMode={optimizationMode}
+          setOptimizationMode={setOptimizationMode}
+          focusMode={focusMode}
+          setFocusMode={setFocusMode}
+          onCancel={handleCancel}
+          systemPromptIds={systemPromptIds}
+          setSystemPromptIds={setSystemPromptIds}
        />
      </div>
-      )}
+      <div ref={messageEnd} className="h-0" />
    </div>
  );
 };
--- a/src/components/ChatWindow.tsx
+++ b/src/components/ChatWindow.tsx
@ -7,12 +7,24 @@ import Chat from './Chat';
 import EmptyChat from './EmptyChat';
 import crypto from 'crypto';
 import { toast } from 'sonner';
-import { useSearchParams } from 'next/navigation';
+import { useSearchParams, useRouter } from 'next/navigation';
 import { getSuggestions } from '@/lib/actions';
 import { Settings } from 'lucide-react';
 import Link from 'next/link';
 import NextError from 'next/error';

+export type ModelStats = {
+  modelName: string;
+  responseTime?: number;
+};
+
+export type AgentActionEvent = {
+  action: string;
+  message: string;
+  details: Record<string, any>;
+  timestamp: Date;
+};
+
 export type Message = {
  messageId: string;
  chatId: string;
@ -21,6 +33,16 @@ export type Message = {
  role: 'user' | 'assistant';
  suggestions?: string[];
  sources?: Document[];
+  modelStats?: ModelStats;
+  searchQuery?: string;
+  searchUrl?: string;
+  agentActions?: AgentActionEvent[];
+  progress?: {
+    message: string;
+    current: number;
+    total: number;
+    subMessage?: string;
+  };
 };

 export interface File {
@ -51,17 +73,6 @@ const checkConfig = async (
    let embeddingModel = localStorage.getItem('embeddingModel');
    let embeddingModelProvider = localStorage.getItem('embeddingModelProvider');

-    const autoImageSearch = localStorage.getItem('autoImageSearch');
-    const autoVideoSearch = localStorage.getItem('autoVideoSearch');
-
-    if (!autoImageSearch) {
-      localStorage.setItem('autoImageSearch', 'true');
-    }
-
-    if (!autoVideoSearch) {
-      localStorage.setItem('autoVideoSearch', 'false');
-    }
-
    const providers = await fetch(`/api/models`, {
      headers: {
        'Content-Type': 'application/json',
@ -266,6 +277,7 @@ const loadMessages = async (

 const ChatWindow = ({ id }: { id?: string }) => {
  const searchParams = useSearchParams();
+  const router = useRouter();
  const initialMessage = searchParams.get('q');

  const [chatId, setChatId] = useState<string | undefined>(id);
@ -299,7 +311,13 @@ const ChatWindow = ({ id }: { id?: string }) => {
  }, []);

  const [loading, setLoading] = useState(false);
-  const [messageAppeared, setMessageAppeared] = useState(false);
+  const [scrollTrigger, setScrollTrigger] = useState(0);
+  const [analysisProgress, setAnalysisProgress] = useState<{
+    message: string;
+    current: number;
+    total: number;
+    subMessage?: string;
+  } | null>(null);

  const [chatHistory, setChatHistory] = useState<[string, string][]>([]);
  const [messages, setMessages] = useState<Message[]>([]);
@ -309,11 +327,22 @@ const ChatWindow = ({ id }: { id?: string }) => {

  const [focusMode, setFocusMode] = useState('webSearch');
  const [optimizationMode, setOptimizationMode] = useState('speed');
+  const [systemPromptIds, setSystemPromptIds] = useState<string[]>([]);

  const [isMessagesLoaded, setIsMessagesLoaded] = useState(false);

  const [notFound, setNotFound] = useState(false);

+  useEffect(() => {
+    const savedOptimizationMode = localStorage.getItem('optimizationMode');
+
+    if (savedOptimizationMode !== null) {
+      setOptimizationMode(savedOptimizationMode);
+    } else {
+      localStorage.setItem('optimizationMode', optimizationMode);
+    }
+  }, [optimizationMode]);
+
  useEffect(() => {
    if (
      chatId &&
@ -354,7 +383,28 @@ const ChatWindow = ({ id }: { id?: string }) => {
    }
  }, [isMessagesLoaded, isConfigReady]);

-  const sendMessage = async (message: string, messageId?: string) => {
+  const sendMessage = async (
+    message: string,
+    options?: {
+      messageId?: string;
+      suggestions?: string[];
+      editMode?: boolean;
+    },
+  ) => {
+    setScrollTrigger((x) => (x === 0 ? -1 : 0));
+    // Special case: If we're just updating an existing message with suggestions
+    if (options?.suggestions && options.messageId) {
+      setMessages((prev) =>
+        prev.map((msg) => {
+          if (msg.messageId === options.messageId) {
+            return { ...msg, suggestions: options.suggestions };
+          }
+          return msg;
+        }),
+      );
+      return;
+    }
+
    if (loading) return;
    if (!isConfigReady) {
      toast.error('Cannot send message before the configuration is ready');
@ -362,13 +412,29 @@ const ChatWindow = ({ id }: { id?: string }) => {
    }

    setLoading(true);
-    setMessageAppeared(false);

    let sources: Document[] | undefined = undefined;
    let recievedMessage = '';
    let added = false;
+    let messageChatHistory = chatHistory;

-    messageId = messageId ?? crypto.randomBytes(7).toString('hex');
+    // If the user is editing or rewriting a message, we need to remove the messages after it
+    const rewriteIndex = messages.findIndex(
+      (msg) => msg.messageId === options?.messageId,
+    );
+    if (rewriteIndex !== -1) {
+      setMessages((prev) => {
+        return [...prev.slice(0, rewriteIndex)];
+      });
+
+      messageChatHistory = chatHistory.slice(0, rewriteIndex);
+      setChatHistory(messageChatHistory);
+
+      setScrollTrigger((prev) => prev + 1);
+    }
+
+    const messageId =
+      options?.messageId ?? crypto.randomBytes(7).toString('hex');

    setMessages((prevMessages) => [
      ...prevMessages,
@ -388,6 +454,46 @@ const ChatWindow = ({ id }: { id?: string }) => {
        return;
      }

+      if (data.type === 'progress') {
+        setAnalysisProgress(data.data);
+        return;
+      }
+
+      // Handle ping messages to keep connection alive (no action needed)
+      if (data.type === 'ping') {
+        console.debug('Ping received');
+        // Ping messages are used to keep the connection alive during long requests
+        // No action is required on the frontend
+        return;
+      }
+
+      if (data.type === 'agent_action') {
+        const agentActionEvent: AgentActionEvent = {
+          action: data.data.action,
+          message: data.data.message,
+          details: data.data.details || {},
+          timestamp: new Date(),
+        };
+
+        // Update the user message with agent actions
+        setMessages((prev) =>
+          prev.map((message) => {
+            if (
+              message.messageId === data.messageId &&
+              message.role === 'user'
+            ) {
+              const updatedActions = [
+                ...(message.agentActions || []),
+                agentActionEvent,
+              ];
+              return { ...message, agentActions: updatedActions };
+            }
+            return message;
+          }),
+        );
+        return;
+      }
+
      if (data.type === 'sources') {
        sources = data.data;
        if (!added) {
@ -399,12 +505,14 @@ const ChatWindow = ({ id }: { id?: string }) => {
              chatId: chatId!,
              role: 'assistant',
              sources: sources,
+              searchQuery: data.searchQuery,
+              searchUrl: data.searchUrl,
              createdAt: new Date(),
            },
          ]);
          added = true;
+          setScrollTrigger((prev) => prev + 1);
        }
-        setMessageAppeared(true);
      }

      if (data.type === 'message') {
@ -418,56 +526,67 @@ const ChatWindow = ({ id }: { id?: string }) => {
              role: 'assistant',
              sources: sources,
              createdAt: new Date(),
+              modelStats: {
+                modelName: data.modelName,
+              },
            },
          ]);
          added = true;
-        }
-
+        } else {
          setMessages((prev) =>
            prev.map((message) => {
              if (message.messageId === data.messageId) {
                return { ...message, content: message.content + data.data };
              }
-
              return message;
            }),
          );
+        }

        recievedMessage += data.data;
-        setMessageAppeared(true);
+        setScrollTrigger((prev) => prev + 1);
      }

      if (data.type === 'messageEnd') {
+        // Clear analysis progress
+        setAnalysisProgress(null);
+
        setChatHistory((prevHistory) => [
          ...prevHistory,
          ['human', message],
          ['assistant', recievedMessage],
        ]);

+        // Always update the message, adding modelStats if available
+        setMessages((prev) =>
+          prev.map((message) => {
+            if (message.messageId === data.messageId) {
+              return {
+                ...message,
+                // Include model stats if available, otherwise null
+                modelStats: data.modelStats || null,
+                // Make sure the searchQuery is preserved (if available in the message data)
+                searchQuery: message.searchQuery || data.searchQuery,
+                searchUrl: message.searchUrl || data.searchUrl,
+              };
+            }
+            return message;
+          }),
+        );
+
        setLoading(false);
+        setScrollTrigger((prev) => prev + 1);

        const lastMsg = messagesRef.current[messagesRef.current.length - 1];

-        const autoImageSearch = localStorage.getItem('autoImageSearch');
-        const autoVideoSearch = localStorage.getItem('autoVideoSearch');
-
-        if (autoImageSearch === 'true') {
-          document
-            .getElementById(`search-images-${lastMsg.messageId}`)
-            ?.click();
-        }
-
-        if (autoVideoSearch === 'true') {
-          document
-            .getElementById(`search-videos-${lastMsg.messageId}`)
-            ?.click();
-        }
+        const autoSuggestions = localStorage.getItem('autoSuggestions');

        if (
          lastMsg.role === 'assistant' &&
          lastMsg.sources &&
          lastMsg.sources.length > 0 &&
-          !lastMsg.suggestions
+          !lastMsg.suggestions &&
+          autoSuggestions !== 'false' // Default to true if not set
        ) {
          const suggestions = await getSuggestions(messagesRef.current);
          setMessages((prev) =>
@ -482,6 +601,21 @@ const ChatWindow = ({ id }: { id?: string }) => {
      }
    };

+    const ollamaContextWindow =
+      localStorage.getItem('ollamaContextWindow') || '2048';
+
+    // Get the latest model selection from localStorage
+    const currentChatModelProvider = localStorage.getItem('chatModelProvider');
+    const currentChatModel = localStorage.getItem('chatModel');
+
+    // Use the most current model selection from localStorage, falling back to the state if not available
+    const modelProvider =
+      currentChatModelProvider || chatModelProvider.provider;
+    const modelName = currentChatModel || chatModelProvider.name;
+
+    const currentOptimizationMode =
+      localStorage.getItem('optimizationMode') || optimizationMode;
+
    const res = await fetch('/api/chat', {
      method: 'POST',
      headers: {
@ -497,17 +631,20 @@ const ChatWindow = ({ id }: { id?: string }) => {
        chatId: chatId!,
        files: fileIds,
        focusMode: focusMode,
-        optimizationMode: optimizationMode,
-        history: chatHistory,
+        optimizationMode: currentOptimizationMode,
+        history: messageChatHistory,
        chatModel: {
-          name: chatModelProvider.name,
-          provider: chatModelProvider.provider,
+          name: modelName,
+          provider: modelProvider,
+          ...(chatModelProvider.provider === 'ollama' && {
+            ollamaContextWindow: parseInt(ollamaContextWindow),
+          }),
        },
        embeddingModel: {
          name: embeddingModelProvider.name,
          provider: embeddingModelProvider.provider,
        },
-        systemInstructions: localStorage.getItem('systemInstructions'),
+        selectedSystemPromptIds: systemPromptIds || [],
      }),
    });

@ -539,25 +676,71 @@ const ChatWindow = ({ id }: { id?: string }) => {
  };

  const rewrite = (messageId: string) => {
-    const index = messages.findIndex((msg) => msg.messageId === messageId);
-
-    if (index === -1) return;
-
-    const message = messages[index - 1];
-
-    setMessages((prev) => {
-      return [...prev.slice(0, messages.length > 2 ? index - 1 : 0)];
-    });
-    setChatHistory((prev) => {
-      return [...prev.slice(0, messages.length > 2 ? index - 1 : 0)];
+    const messageIndex = messages.findIndex(
+      (msg) => msg.messageId === messageId,
+    );
+    if (messageIndex == -1) return;
+    sendMessage(messages[messageIndex - 1].content, {
+      messageId: messages[messageIndex - 1].messageId,
    });
+  };

-    sendMessage(message.content, message.messageId);
+  const handleEditMessage = async (messageId: string, newContent: string) => {
+    // Get the index of the message being edited
+    const messageIndex = messages.findIndex(
+      (msg) => msg.messageId === messageId,
+    );
+    if (messageIndex === -1) return;
+
+    try {
+      sendMessage(newContent, {
+        messageId,
+        editMode: true,
+      });
+    } catch (error) {
+      console.error('Error updating message:', error);
+      toast.error('Failed to update message');
+    }
  };

  useEffect(() => {
    if (isReady && initialMessage && isConfigReady) {
+      // Check if we have an initial query and apply saved search settings
+      const searchOptimizationMode = localStorage.getItem(
+        'searchOptimizationMode',
+      );
+      const searchChatModelProvider = localStorage.getItem(
+        'searchChatModelProvider',
+      );
+      const searchChatModel = localStorage.getItem('searchChatModel');
+
+      // Apply saved optimization mode if valid
+      if (
+        searchOptimizationMode &&
+        (searchOptimizationMode === 'speed' ||
+          searchOptimizationMode === 'agent')
+      ) {
+        setOptimizationMode(searchOptimizationMode);
+        localStorage.setItem('optimizationMode', searchOptimizationMode);
+      }
+
+      // Apply saved chat model if valid
+      if (searchChatModelProvider && searchChatModel) {
+        setChatModelProvider({
+          name: searchChatModel,
+          provider: searchChatModelProvider,
+        });
+        // Also update localStorage to ensure consistency
+        localStorage.setItem('chatModelProvider', searchChatModelProvider);
+        localStorage.setItem('chatModel', searchChatModel);
+      }
+
      sendMessage(initialMessage);
+
+      // Remove the query parameter from the URL to prevent re-execution on page reload
+      const url = new URL(window.location.href);
+      url.searchParams.delete('q');
+      router.replace(url.pathname + url.search, { scroll: false });
    }
    // eslint-disable-next-line react-hooks/exhaustive-deps
  }, [isConfigReady, isReady, initialMessage]);
@ -591,12 +774,20 @@ const ChatWindow = ({ id }: { id?: string }) => {
              loading={loading}
              messages={messages}
              sendMessage={sendMessage}
-              messageAppeared={messageAppeared}
+              scrollTrigger={scrollTrigger}
              rewrite={rewrite}
              fileIds={fileIds}
              setFileIds={setFileIds}
              files={files}
              setFiles={setFiles}
+              optimizationMode={optimizationMode}
+              setOptimizationMode={setOptimizationMode}
+              focusMode={focusMode}
+              setFocusMode={setFocusMode}
+              handleEditMessage={handleEditMessage}
+              analysisProgress={analysisProgress}
+              systemPromptIds={systemPromptIds}
+              setSystemPromptIds={setSystemPromptIds}
            />
          </>
        ) : (
@ -606,6 +797,8 @@ const ChatWindow = ({ id }: { id?: string }) => {
            setFocusMode={setFocusMode}
            optimizationMode={optimizationMode}
            setOptimizationMode={setOptimizationMode}
+            systemPromptIds={systemPromptIds}
+            setSystemPromptIds={setSystemPromptIds}
            fileIds={fileIds}
            setFileIds={setFileIds}
            files={files}
--- a/src/components/EmptyChat.tsx
+++ b/src/components/EmptyChat.tsx
@ -1,7 +1,8 @@
 import { Settings } from 'lucide-react';
-import EmptyChatMessageInput from './EmptyChatMessageInput';
+import { useState } from 'react';
 import { File } from './ChatWindow';
 import Link from 'next/link';
+import MessageInput from './MessageInput';
 import WeatherWidget from './WeatherWidget';
 import NewsArticleWidget from './NewsArticleWidget';

@ -11,6 +12,8 @@ const EmptyChat = ({
  setFocusMode,
  optimizationMode,
  setOptimizationMode,
+  systemPromptIds,
+  setSystemPromptIds,
  fileIds,
  setFileIds,
  files,
@ -21,6 +24,8 @@ const EmptyChat = ({
  setFocusMode: (mode: string) => void;
  optimizationMode: string;
  setOptimizationMode: (mode: string) => void;
+  systemPromptIds: string[];
+  setSystemPromptIds: (ids: string[]) => void;
  fileIds: string[];
  setFileIds: (fileIds: string[]) => void;
  files: File[];
@ -38,7 +43,9 @@ const EmptyChat = ({
          <h2 className="text-black/70 dark:text-white/70 text-3xl font-medium -mt-8">
            Research begins here.
          </h2>
-          <EmptyChatMessageInput
+          <MessageInput
+            firstMessage={true}
+            loading={false}
            sendMessage={sendMessage}
            focusMode={focusMode}
            setFocusMode={setFocusMode}
@ -47,6 +54,8 @@ const EmptyChat = ({
            fileIds={fileIds}
            setFileIds={setFileIds}
            files={files}
+            systemPromptIds={systemPromptIds}
+            setSystemPromptIds={setSystemPromptIds}
            setFiles={setFiles}
          />
        </div>
--- a/src/components/EmptyChatMessageInput.tsx
+++ b/src/components/EmptyChatMessageInput.tsx
@ -1,114 +0,0 @@
-import { ArrowRight } from 'lucide-react';
-import { useEffect, useRef, useState } from 'react';
-import TextareaAutosize from 'react-textarea-autosize';
-import CopilotToggle from './MessageInputActions/Copilot';
-import Focus from './MessageInputActions/Focus';
-import Optimization from './MessageInputActions/Optimization';
-import Attach from './MessageInputActions/Attach';
-import { File } from './ChatWindow';
-
-const EmptyChatMessageInput = ({
-  sendMessage,
-  focusMode,
-  setFocusMode,
-  optimizationMode,
-  setOptimizationMode,
-  fileIds,
-  setFileIds,
-  files,
-  setFiles,
-}: {
-  sendMessage: (message: string) => void;
-  focusMode: string;
-  setFocusMode: (mode: string) => void;
-  optimizationMode: string;
-  setOptimizationMode: (mode: string) => void;
-  fileIds: string[];
-  setFileIds: (fileIds: string[]) => void;
-  files: File[];
-  setFiles: (files: File[]) => void;
-}) => {
-  const [copilotEnabled, setCopilotEnabled] = useState(false);
-  const [message, setMessage] = useState('');
-
-  const inputRef = useRef<HTMLTextAreaElement | null>(null);
-
-  useEffect(() => {
-    const handleKeyDown = (e: KeyboardEvent) => {
-      const activeElement = document.activeElement;
-
-      const isInputFocused =
-        activeElement?.tagName === 'INPUT' ||
-        activeElement?.tagName === 'TEXTAREA' ||
-        activeElement?.hasAttribute('contenteditable');
-
-      if (e.key === '/' && !isInputFocused) {
-        e.preventDefault();
-        inputRef.current?.focus();
-      }
-    };
-
-    document.addEventListener('keydown', handleKeyDown);
-
-    inputRef.current?.focus();
-
-    return () => {
-      document.removeEventListener('keydown', handleKeyDown);
-    };
-  }, []);
-
-  return (
-    <form
-      onSubmit={(e) => {
-        e.preventDefault();
-        sendMessage(message);
-        setMessage('');
-      }}
-      onKeyDown={(e) => {
-        if (e.key === 'Enter' && !e.shiftKey) {
-          e.preventDefault();
-          sendMessage(message);
-          setMessage('');
-        }
-      }}
-      className="w-full"
-    >
-      <div className="flex flex-col bg-light-secondary dark:bg-dark-secondary px-5 pt-5 pb-2 rounded-lg w-full border border-light-200 dark:border-dark-200">
-        <TextareaAutosize
-          ref={inputRef}
-          value={message}
-          onChange={(e) => setMessage(e.target.value)}
-          minRows={2}
-          className="bg-transparent placeholder:text-black/50 dark:placeholder:text-white/50 text-sm text-black dark:text-white resize-none focus:outline-none w-full max-h-24 lg:max-h-36 xl:max-h-48"
-          placeholder="Ask anything..."
-        />
-        <div className="flex flex-row items-center justify-between mt-4">
-          <div className="flex flex-row items-center space-x-2 lg:space-x-4">
-            <Focus focusMode={focusMode} setFocusMode={setFocusMode} />
-            <Attach
-              fileIds={fileIds}
-              setFileIds={setFileIds}
-              files={files}
-              setFiles={setFiles}
-              showText
-            />
-          </div>
-          <div className="flex flex-row items-center space-x-1 sm:space-x-4">
-            <Optimization
-              optimizationMode={optimizationMode}
-              setOptimizationMode={setOptimizationMode}
-            />
-            <button
-              disabled={message.trim().length === 0}
-              className="bg-[#24A0ED] text-white disabled:text-black/50 dark:disabled:text-white/50 disabled:bg-[#e0e0dc] dark:disabled:bg-[#ececec21] hover:bg-opacity-85 transition duration-100 rounded-full p-2"
-            >
-              <ArrowRight className="bg-background" size={17} />
-            </button>
-          </div>
-        </div>
-      </div>
-    </form>
-  );
-};
-
-export default EmptyChatMessageInput;
--- a/src/components/MessageActions/ModelInfo.tsx
+++ b/src/components/MessageActions/ModelInfo.tsx
@ -0,0 +1,82 @@
+'use client';
+
+import React, { useState, useEffect, useRef } from 'react';
+import { Info } from 'lucide-react';
+import { ModelStats } from '../ChatWindow';
+import { cn } from '@/lib/utils';
+
+interface ModelInfoButtonProps {
+  modelStats: ModelStats | null;
+}
+
+const ModelInfoButton: React.FC<ModelInfoButtonProps> = ({ modelStats }) => {
+  const [showPopover, setShowPopover] = useState(false);
+  const popoverRef = useRef<HTMLDivElement>(null);
+  const buttonRef = useRef<HTMLButtonElement>(null);
+
+  // Always render, using "Unknown" as fallback if model info isn't available
+  const modelName = modelStats?.modelName || 'Unknown';
+
+  useEffect(() => {
+    const handleClickOutside = (event: MouseEvent) => {
+      if (
+        popoverRef.current &&
+        !popoverRef.current.contains(event.target as Node) &&
+        buttonRef.current &&
+        !buttonRef.current.contains(event.target as Node)
+      ) {
+        setShowPopover(false);
+      }
+    };
+
+    document.addEventListener('mousedown', handleClickOutside);
+    return () => {
+      document.removeEventListener('mousedown', handleClickOutside);
+    };
+  }, []);
+
+  return (
+    <div className="relative">
+      <button
+        ref={buttonRef}
+        className="p-1 ml-1 text-black/70 dark:text-white/70 rounded-full hover:bg-light-secondary dark:hover:bg-dark-secondary transition duration-200 hover:text-black dark:hover:text-white"
+        onClick={() => setShowPopover(!showPopover)}
+        aria-label="Show model information"
+      >
+        <Info size={18} />
+      </button>
+      {showPopover && (
+        <div
+          ref={popoverRef}
+          className="absolute z-10 left-6 top-0 w-64 rounded-md shadow-lg bg-white dark:bg-dark-secondary border border-light-200 dark:border-dark-200"
+        >
+          <div className="py-2 px-3">
+            <h4 className="text-sm font-medium mb-2 text-black dark:text-white">
+              Model Information
+            </h4>
+            <div className="space-y-1 text-xs">
+              <div className="flex justify-between">
+                <span className="text-black/70 dark:text-white/70">Model:</span>
+                <span className="text-black dark:text-white font-medium">
+                  {modelName}
+                </span>
+              </div>
+              {modelStats?.responseTime && (
+                <div className="flex justify-between">
+                  <span className="text-black/70 dark:text-white/70">
+                    Response time:
+                  </span>
+                  <span className="text-black dark:text-white font-medium">
+                    {(modelStats.responseTime / 1000).toFixed(2)}s
+                  </span>
+                </div>
+              )}
+            </div>
+          </div>
+        </div>
+      )}
+    </div>
+  );
+};
+
+export default ModelInfoButton;
--- a/src/components/MessageBox.tsx
+++ b/src/components/MessageBox.tsx
@ -1,120 +1,56 @@
-'use client';
-
-/* eslint-disable @next/next/no-img-element */
-import React, { MutableRefObject, useEffect, useState } from 'react';
-import { Message } from './ChatWindow';
 import { cn } from '@/lib/utils';
-import {
-  BookCopy,
-  Disc3,
-  Volume2,
-  StopCircle,
-  Layers3,
-  Plus,
-} from 'lucide-react';
-import Markdown, { MarkdownToJSX } from 'markdown-to-jsx';
-import Copy from './MessageActions/Copy';
-import Rewrite from './MessageActions/Rewrite';
-import MessageSources from './MessageSources';
-import SearchImages from './SearchImages';
-import SearchVideos from './SearchVideos';
-import { useSpeech } from 'react-text-to-speech';
-import ThinkBox from './ThinkBox';
-
-const ThinkTagProcessor = ({ children }: { children: React.ReactNode }) => {
-  return <ThinkBox content={children as string} />;
-};
+import { Check, Pencil, X } from 'lucide-react';
+import { useState } from 'react';
+import { Message } from './ChatWindow';
+import MessageTabs from './MessageTabs';

 const MessageBox = ({
  message,
  messageIndex,
  history,
  loading,
-  dividerRef,
  isLast,
  rewrite,
  sendMessage,
+  handleEditMessage,
 }: {
  message: Message;
  messageIndex: number;
  history: Message[];
  loading: boolean;
-  dividerRef?: MutableRefObject<HTMLDivElement | null>;
  isLast: boolean;
  rewrite: (messageId: string) => void;
-  sendMessage: (message: string) => void;
+  sendMessage: (
+    message: string,
+    options?: {
+      messageId?: string;
+      rewriteIndex?: number;
+      suggestions?: string[];
+    },
+  ) => void;
+  handleEditMessage: (messageId: string, content: string) => void;
 }) => {
-  const [parsedMessage, setParsedMessage] = useState(message.content);
-  const [speechMessage, setSpeechMessage] = useState(message.content);
+  // Local state for editing functionality
+  const [isEditing, setIsEditing] = useState(false);
+  const [editedContent, setEditedContent] = useState('');

-  useEffect(() => {
-    const citationRegex = /\[([^\]]+)\]/g;
-    const regex = /\[(\d+)\]/g;
-    let processedMessage = message.content;
-
-    if (message.role === 'assistant' && message.content.includes('<think>')) {
-      const openThinkTag = processedMessage.match(/<think>/g)?.length || 0;
-      const closeThinkTag = processedMessage.match(/<\/think>/g)?.length || 0;
-
-      if (openThinkTag > closeThinkTag) {
-        processedMessage += '</think> <a> </a>'; // The extra <a> </a> is to prevent the the think component from looking bad
-      }
-    }
-
-    if (
-      message.role === 'assistant' &&
-      message?.sources &&
-      message.sources.length > 0
-    ) {
-      setParsedMessage(
-        processedMessage.replace(
-          citationRegex,
-          (_, capturedContent: string) => {
-            const numbers = capturedContent
-              .split(',')
-              .map((numStr) => numStr.trim());
-
-            const linksHtml = numbers
-              .map((numStr) => {
-                const number = parseInt(numStr);
-
-                if (isNaN(number) || number <= 0) {
-                  return `[${numStr}]`;
-                }
-
-                const source = message.sources?.[number - 1];
-                const url = source?.metadata?.url;
-
-                if (url) {
-                  return `<a href="${url}" target="_blank" className="bg-light-secondary dark:bg-dark-secondary px-1 rounded ml-1 no-underline text-xs text-black/70 dark:text-white/70 relative">${numStr}</a>`;
-                } else {
-                  return `[${numStr}]`;
-                }
-              })
-              .join('');
-
-            return linksHtml;
-          },
-        ),
-      );
-      setSpeechMessage(message.content.replace(regex, ''));
-      return;
-    }
-
-    setSpeechMessage(message.content.replace(regex, ''));
-    setParsedMessage(processedMessage);
-  }, [message.content, message.sources, message.role]);
-
-  const { speechStatus, start, stop } = useSpeech({ text: speechMessage });
-
-  const markdownOverrides: MarkdownToJSX.Options = {
-    overrides: {
-      think: {
-        component: ThinkTagProcessor,
-      },
-    },
+  // Initialize editing
+  const startEditMessage = () => {
+    setIsEditing(true);
+    setEditedContent(message.content);
  };

+  // Cancel editing
+  const cancelEditMessage = () => {
+    setIsEditing(false);
+    setEditedContent('');
+  };
+
+  // Save edits
+  const saveEditMessage = () => {
+    handleEditMessage(message.messageId, editedContent);
+    setIsEditing(false);
+  };
  return (
    <div>
      {message.role === 'user' && (
@ -125,135 +61,69 @@ const MessageBox = ({
            'break-words',
          )}
        >
-          <h2 className="text-black dark:text-white font-medium text-3xl lg:w-9/12">
-            {message.content}
-          </h2>
-        </div>
-      )}
-
-      {message.role === 'assistant' && (
-        <div className="flex flex-col space-y-9 lg:space-y-0 lg:flex-row lg:justify-between lg:space-x-9">
-          <div
-            ref={dividerRef}
-            className="flex flex-col space-y-6 w-full lg:w-9/12"
-          >
-            {message.sources && message.sources.length > 0 && (
-              <div className="flex flex-col space-y-2">
-                <div className="flex flex-row items-center space-x-2">
-                  <BookCopy className="text-black dark:text-white" size={20} />
-                  <h3 className="text-black dark:text-white font-medium text-xl">
-                    Sources
-                  </h3>
-                </div>
-                <MessageSources sources={message.sources} />
-              </div>
-            )}
-            <div className="flex flex-col space-y-2">
-              <div className="flex flex-row items-center space-x-2">
-                <Disc3
-                  className={cn(
-                    'text-black dark:text-white',
-                    isLast && loading ? 'animate-spin' : 'animate-none',
-                  )}
-                  size={20}
+          {isEditing ? (
+            <div className="w-full">
+              <textarea
+                className="w-full p-3 text-lg bg-light-100 dark:bg-dark-100 rounded-lg border border-light-secondary dark:border-dark-secondary text-black dark:text-white focus:outline-none focus:border-[#24A0ED] transition duration-200 min-h-[120px] font-medium"
+                value={editedContent}
+                onChange={(e) => setEditedContent(e.target.value)}
+                placeholder="Edit your message..."
+                autoFocus
              />
-                <h3 className="text-black dark:text-white font-medium text-xl">
-                  Answer
-                </h3>
-              </div>
-
-              <Markdown
-                className={cn(
-                  'prose prose-h1:mb-3 prose-h2:mb-2 prose-h2:mt-6 prose-h2:font-[800] prose-h3:mt-4 prose-h3:mb-1.5 prose-h3:font-[600] dark:prose-invert prose-p:leading-relaxed prose-pre:p-0 font-[400]',
-                  'max-w-none break-words text-black dark:text-white',
-                )}
-                options={markdownOverrides}
-              >
-                {parsedMessage}
-              </Markdown>
-              {loading && isLast ? null : (
-                <div className="flex flex-row items-center justify-between w-full text-black dark:text-white py-4 -mx-2">
-                  <div className="flex flex-row items-center space-x-1">
-                    {/*  <button className="p-2 text-black/70 dark:text-white/70 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary transition duration-200 hover:text-black text-black dark:hover:text-white">
-                      <Share size={18} />
-                    </button> */}
-                    <Rewrite rewrite={rewrite} messageId={message.messageId} />
-                  </div>
-                  <div className="flex flex-row items-center space-x-1">
-                    <Copy initialMessage={message.content} message={message} />
+              <div className="flex flex-row space-x-2 mt-3 justify-end">
                <button
-                      onClick={() => {
-                        if (speechStatus === 'started') {
-                          stop();
-                        } else {
-                          start();
-                        }
-                      }}
-                      className="p-2 text-black/70 dark:text-white/70 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary transition duration-200 hover:text-black dark:hover:text-white"
+                  onClick={cancelEditMessage}
+                  className="p-2 rounded-full bg-light-secondary dark:bg-dark-secondary hover:bg-light-200 dark:hover:bg-dark-200 transition duration-200 text-black/70 dark:text-white/70 hover:text-black dark:hover:text-white"
+                  aria-label="Cancel"
+                  title="Cancel"
                >
-                      {speechStatus === 'started' ? (
-                        <StopCircle size={18} />
-                      ) : (
-                        <Volume2 size={18} />
-                      )}
+                  <X size={18} />
+                </button>
+                <button
+                  onClick={saveEditMessage}
+                  className="p-2 rounded-full bg-[#24A0ED] hover:bg-[#1a8ad3] transition duration-200 text-white disabled:opacity-50 disabled:cursor-not-allowed"
+                  aria-label="Save changes"
+                  title="Save changes"
+                  disabled={!editedContent.trim()}
+                >
+                  <Check size={18} className="text-white" />
                </button>
              </div>
            </div>
-              )}
-              {isLast &&
-                message.suggestions &&
-                message.suggestions.length > 0 &&
-                message.role === 'assistant' &&
-                !loading && (
+          ) : (
            <>
-                    <div className="h-px w-full bg-light-secondary dark:bg-dark-secondary" />
-                    <div className="flex flex-col space-y-3 text-black dark:text-white">
-                      <div className="flex flex-row items-center space-x-2 mt-4">
-                        <Layers3 />
-                        <h3 className="text-xl font-medium">Related</h3>
-                      </div>
-                      <div className="flex flex-col space-y-3">
-                        {message.suggestions.map((suggestion, i) => (
-                          <div
-                            className="flex flex-col space-y-3 text-sm"
-                            key={i}
+              <div className="flex items-center">
+                <h2
+                  className="text-black dark:text-white font-medium text-3xl"
+                  onClick={startEditMessage}
                >
-                            <div className="h-px w-full bg-light-secondary dark:bg-dark-secondary" />
-                            <div
-                              onClick={() => {
-                                sendMessage(suggestion);
-                              }}
-                              className="cursor-pointer flex flex-row justify-between font-medium space-x-2 items-center"
+                  {message.content}
+                </h2>
+                <button
+                  onClick={startEditMessage}
+                  className="ml-3 p-2 rounded-xl bg-light-secondary dark:bg-dark-secondary text-black/70 dark:text-white/70 hover:text-black dark:hover:text-white flex-shrink-0"
+                  aria-label="Edit message"
+                  title="Edit message"
                >
-                              <p className="transition duration-200 hover:text-[#24A0ED]">
-                                {suggestion}
-                              </p>
-                              <Plus
-                                size={20}
-                                className="text-[#24A0ED] flex-shrink-0"
-                              />
-                            </div>
-                          </div>
-                        ))}
-                      </div>
+                  <Pencil size={18} />
+                </button>
              </div>
            </>
          )}
        </div>
-          </div>
-          <div className="lg:sticky lg:top-20 flex flex-col items-center space-y-3 w-full lg:w-3/12 z-30 h-full pb-4">
-            <SearchImages
+      )}
+
+      {message.role === 'assistant' && (
+        <MessageTabs
          query={history[messageIndex - 1].content}
          chatHistory={history.slice(0, messageIndex - 1)}
          messageId={message.messageId}
+          message={message}
+          isLast={isLast}
+          loading={loading}
+          rewrite={rewrite}
+          sendMessage={sendMessage}
        />
-            <SearchVideos
-              chatHistory={history.slice(0, messageIndex - 1)}
-              query={history[messageIndex - 1].content}
-              messageId={message.messageId}
-            />
-          </div>
-        </div>
      )}
    </div>
  );
--- a/src/components/MessageBoxLoading.tsx
+++ b/src/components/MessageBoxLoading.tsx
@ -1,9 +1,50 @@
-const MessageBoxLoading = () => {
+interface MessageBoxLoadingProps {
+  progress: {
+    message: string;
+    current: number;
+    total: number;
+    subMessage?: string;
+  } | null;
+}
+
+const MessageBoxLoading = ({ progress }: MessageBoxLoadingProps) => {
  return (
-    <div className="flex flex-col space-y-2 w-full lg:w-9/12 bg-light-primary dark:bg-dark-primary animate-pulse rounded-lg py-3">
-      <div className="h-2 rounded-full w-full bg-light-secondary dark:bg-dark-secondary" />
-      <div className="h-2 rounded-full w-9/12 bg-light-secondary dark:bg-dark-secondary" />
-      <div className="h-2 rounded-full w-10/12 bg-light-secondary dark:bg-dark-secondary" />
+    <div className="flex flex-col space-y-4 w-full lg:w-9/12">
+      {progress && progress.current !== progress.total ? (
+        <div className="bg-light-primary dark:bg-dark-primary rounded-lg p-4">
+          <div className="flex flex-col space-y-3">
+            <p className="text-base font-semibold text-black dark:text-white">
+              {progress.message}
+            </p>
+            {progress.subMessage && (
+              <p
+                className="text-xs text-black/40 dark:text-white/40 mt-1"
+                title={progress.subMessage}
+              >
+                {progress.subMessage}
+              </p>
+            )}
+            <div className="w-full bg-light-secondary dark:bg-dark-secondary rounded-full h-2 overflow-hidden">
+              <div
+                className={`h-full bg-[#24A0ED] transition-all duration-300 ease-in-out ${
+                  progress.current === progress.total ? '' : 'animate-pulse'
+                }`}
+                style={{
+                  width: `${(progress.current / progress.total) * 100}%`,
+                }}
+              />
+            </div>
+          </div>
+        </div>
+      ) : (
+        <div className="pl-3 flex items-center justify-start">
+          <div className="flex space-x-1">
+            <div className="w-1.5 h-1.5 bg-black/40 dark:bg-white/40 rounded-full animate-[high-bounce_1s_infinite] [animation-delay:-0.3s]"></div>
+            <div className="w-1.5 h-1.5 bg-black/40 dark:bg-white/40 rounded-full animate-[high-bounce_1s_infinite] [animation-delay:-0.15s]"></div>
+            <div className="w-1.5 h-1.5 bg-black/40 dark:bg-white/40 rounded-full animate-[high-bounce_1s_infinite]"></div>
+          </div>
+        </div>
+      )}
    </div>
  );
 };
--- a/src/components/MessageInput.tsx
+++ b/src/components/MessageInput.tsx
@ -1,11 +1,12 @@
-import { cn } from '@/lib/utils';
-import { ArrowUp } from 'lucide-react';
+import { ArrowRight, ArrowUp, Square } from 'lucide-react';
 import { useEffect, useRef, useState } from 'react';
 import TextareaAutosize from 'react-textarea-autosize';
-import Attach from './MessageInputActions/Attach';
-import CopilotToggle from './MessageInputActions/Copilot';
 import { File } from './ChatWindow';
-import AttachSmall from './MessageInputActions/AttachSmall';
+import Attach from './MessageInputActions/Attach';
+import Focus from './MessageInputActions/Focus';
+import ModelSelector from './MessageInputActions/ModelSelector';
+import Optimization from './MessageInputActions/Optimization';
+import SystemPromptSelector from './MessageInputActions/SystemPromptSelector'; // Import new component

 const MessageInput = ({
  sendMessage,
@ -14,125 +15,208 @@ const MessageInput = ({
  setFileIds,
  files,
  setFiles,
+  optimizationMode,
+  setOptimizationMode,
+  focusMode,
+  setFocusMode,
+  firstMessage,
+  onCancel,
+  systemPromptIds,
+  setSystemPromptIds,
 }: {
-  sendMessage: (message: string) => void;
+  sendMessage: (
+    message: string,
+    options?: {
+      messageId?: string; // For rewrites/edits
+      selectedSystemPromptIds?: string[];
+    },
+  ) => void;
  loading: boolean;
  fileIds: string[];
  setFileIds: (fileIds: string[]) => void;
  files: File[];
  setFiles: (files: File[]) => void;
+  optimizationMode: string;
+  setOptimizationMode: (mode: string) => void;
+  focusMode: string;
+  setFocusMode: (mode: string) => void;
+  firstMessage: boolean;
+  onCancel?: () => void;
+  systemPromptIds: string[];
+  setSystemPromptIds: (ids: string[]) => void;
 }) => {
-  const [copilotEnabled, setCopilotEnabled] = useState(false);
  const [message, setMessage] = useState('');
-  const [textareaRows, setTextareaRows] = useState(1);
-  const [mode, setMode] = useState<'multi' | 'single'>('single');
+  const [selectedModel, setSelectedModel] = useState<{
+    provider: string;
+    model: string;
+  } | null>(null);

  useEffect(() => {
-    if (textareaRows >= 2 && message && mode === 'single') {
-      setMode('multi');
-    } else if (!message && mode === 'multi') {
-      setMode('single');
+    // Load saved model preferences from localStorage
+    const chatModelProvider = localStorage.getItem('chatModelProvider');
+    const chatModel = localStorage.getItem('chatModel');
+
+    if (chatModelProvider && chatModel) {
+      setSelectedModel({
+        provider: chatModelProvider,
+        model: chatModel,
+      });
    }
-  }, [textareaRows, mode, message]);
+
+    const storedPromptIds = localStorage.getItem('selectedSystemPromptIds');
+    if (storedPromptIds) {
+      try {
+        const parsedIds = JSON.parse(storedPromptIds);
+        if (Array.isArray(parsedIds)) {
+          setSystemPromptIds(parsedIds);
+        }
+      } catch (e) {
+        console.error(
+          'Failed to parse selectedSystemPromptIds from localStorage',
+          e,
+        );
+        localStorage.removeItem('selectedSystemPromptIds'); // Clear corrupted data
+      }
+    }
+  }, [setSystemPromptIds]);
+
+  useEffect(() => {
+    if (systemPromptIds.length > 0) {
+      localStorage.setItem(
+        'selectedSystemPromptIds',
+        JSON.stringify(systemPromptIds),
+      );
+    } else {
+      // Remove from localStorage if no prompts are selected to keep it clean
+      localStorage.removeItem('selectedSystemPromptIds');
+    }
+  }, [systemPromptIds]);

  const inputRef = useRef<HTMLTextAreaElement | null>(null);

  useEffect(() => {
    const handleKeyDown = (e: KeyboardEvent) => {
      const activeElement = document.activeElement;
-
      const isInputFocused =
        activeElement?.tagName === 'INPUT' ||
        activeElement?.tagName === 'TEXTAREA' ||
        activeElement?.hasAttribute('contenteditable');
-
      if (e.key === '/' && !isInputFocused) {
        e.preventDefault();
        inputRef.current?.focus();
      }
    };
-
    document.addEventListener('keydown', handleKeyDown);
-
    return () => {
      document.removeEventListener('keydown', handleKeyDown);
    };
  }, []);

+  // Function to handle message submission
+  const handleSubmitMessage = () => {
+    // Only submit if we have a non-empty message and not currently loading
+    if (loading || message.trim().length === 0) return;
+
+    // Make sure the selected model is used when sending a message
+    if (selectedModel) {
+      localStorage.setItem('chatModelProvider', selectedModel.provider);
+      localStorage.setItem('chatModel', selectedModel.model);
+    }
+
+    sendMessage(message);
+    setMessage('');
+  };
+
  return (
    <form
      onSubmit={(e) => {
-        if (loading) return;
        e.preventDefault();
-        sendMessage(message);
-        setMessage('');
+        handleSubmitMessage();
      }}
      onKeyDown={(e) => {
-        if (e.key === 'Enter' && !e.shiftKey && !loading) {
+        if (e.key === 'Enter' && !e.shiftKey) {
          e.preventDefault();
-          sendMessage(message);
-          setMessage('');
+          handleSubmitMessage();
        }
      }}
-      className={cn(
-        'bg-light-secondary dark:bg-dark-secondary p-4 flex items-center overflow-hidden border border-light-200 dark:border-dark-200',
-        mode === 'multi' ? 'flex-col rounded-lg' : 'flex-row rounded-full',
-      )}
+      className="w-full"
    >
-      {mode === 'single' && (
-        <AttachSmall
-          fileIds={fileIds}
-          setFileIds={setFileIds}
-          files={files}
-          setFiles={setFiles}
-        />
-      )}
+      <div className="flex flex-col bg-light-secondary dark:bg-dark-secondary px-3 pt-4 pb-2 rounded-lg w-full border border-light-200 dark:border-dark-200">
+        <div className="flex flex-row items-end space-x-2 mb-2">
          <TextareaAutosize
            ref={inputRef}
            value={message}
            onChange={(e) => setMessage(e.target.value)}
-        onHeightChange={(height, props) => {
-          setTextareaRows(Math.ceil(height / props.rowHeight));
+            minRows={1}
+            className="mb-2 bg-transparent placeholder:text-black/50 dark:placeholder:text-white/50 text-sm text-black dark:text-white resize-none focus:outline-none w-full max-h-24 lg:max-h-36 xl:max-h-48"
+            placeholder={firstMessage ? 'Ask anything...' : 'Ask a follow-up'}
+          />
+          <Optimization
+            optimizationMode={optimizationMode}
+            setOptimizationMode={(optimizationMode) => {
+              setOptimizationMode(optimizationMode);
+              localStorage.setItem('optimizationMode', optimizationMode);
            }}
-        className="transition bg-transparent dark:placeholder:text-white/50 placeholder:text-sm text-sm dark:text-white resize-none focus:outline-none w-full px-2 max-h-24 lg:max-h-36 xl:max-h-48 flex-grow flex-shrink"
-        placeholder="Ask a follow-up"
          />
-      {mode === 'single' && (
-        <div className="flex flex-row items-center space-x-4">
-          <CopilotToggle
-            copilotEnabled={copilotEnabled}
-            setCopilotEnabled={setCopilotEnabled}
-          />
-          <button
-            disabled={message.trim().length === 0 || loading}
-            className="bg-[#24A0ED] text-white disabled:text-black/50 dark:disabled:text-white/50 hover:bg-opacity-85 transition duration-100 disabled:bg-[#e0e0dc79] dark:disabled:bg-[#ececec21] rounded-full p-2"
-          >
-            <ArrowUp className="bg-background" size={17} />
-          </button>
        </div>
-      )}
-      {mode === 'multi' && (
-        <div className="flex flex-row items-center justify-between w-full pt-2">
-          <AttachSmall
+        <div className="flex flex-row items-center justify-between">
+          <div className="flex flex-row items-center space-x-2">
+            <Focus focusMode={focusMode} setFocusMode={setFocusMode} />
+            <Attach
              fileIds={fileIds}
              setFileIds={setFileIds}
              files={files}
              setFiles={setFiles}
+              optimizationMode={optimizationMode}
            />
-          <div className="flex flex-row items-center space-x-4">
-            <CopilotToggle
-              copilotEnabled={copilotEnabled}
-              setCopilotEnabled={setCopilotEnabled}
+          </div>
+          <div className="flex flex-row items-center space-x-2">
+            <ModelSelector
+              showModelName={false}
+              selectedModel={selectedModel}
+              setSelectedModel={(selectedModel) => {
+                setSelectedModel(selectedModel);
+                localStorage.setItem(
+                  'chatModelProvider',
+                  selectedModel.provider,
+                );
+                localStorage.setItem('chatModel', selectedModel.model);
+              }}
            />
+            <SystemPromptSelector
+              selectedPromptIds={systemPromptIds}
+              onSelectedPromptIdsChange={setSystemPromptIds}
+            />
+            {loading ? (
              <button
-              disabled={message.trim().length === 0 || loading}
-              className="bg-[#24A0ED] text-white text-black/50 dark:disabled:text-white/50 hover:bg-opacity-85 transition duration-100 disabled:bg-[#e0e0dc79] dark:disabled:bg-[#ececec21] rounded-full p-2"
+                type="button"
+                className="bg-red-700 text-white hover:bg-red-800 transition duration-100 rounded-full p-2 relative group"
+                onClick={onCancel}
+                aria-label="Cancel"
              >
-              <ArrowUp className="bg-background" size={17} />
-            </button>
-          </div>
-        </div>
+                {loading && (
+                  <div className="absolute inset-0 rounded-full border-2 border-white/30 border-t-white animate-spin" />
                )}
+                <span className="relative flex items-center justify-center w-[17px] h-[17px]">
+                  <Square size={17} className="text-white" />
+                </span>
+              </button>
+            ) : (
+              <button
+                disabled={message.trim().length === 0}
+                className="bg-[#24A0ED] text-white disabled:text-black/50 dark:disabled:text-white/50 disabled:bg-[#e0e0dc] dark:disabled:bg-[#ececec21] hover:bg-opacity-85 transition duration-100 rounded-full p-2"
+                type="submit"
+              >
+                {firstMessage ? (
+                  <ArrowRight className="bg-background" size={17} />
+                ) : (
+                  <ArrowUp className="bg-background" size={17} />
+                )}
+              </button>
+            )}
+          </div>
+        </div>
+      </div>
    </form>
  );
 };
--- a/src/components/MessageInputActions/Attach.tsx
+++ b/src/components/MessageInputActions/Attach.tsx
@ -5,27 +5,32 @@ import {
  PopoverPanel,
  Transition,
 } from '@headlessui/react';
-import { CopyPlus, File, LoaderCircle, Plus, Trash } from 'lucide-react';
+import { File, LoaderCircle, Paperclip, Plus, Trash } from 'lucide-react';
 import { Fragment, useRef, useState } from 'react';
 import { File as FileType } from '../ChatWindow';

 const Attach = ({
  fileIds,
  setFileIds,
-  showText,
  files,
  setFiles,
+  optimizationMode,
 }: {
  fileIds: string[];
  setFileIds: (fileIds: string[]) => void;
-  showText?: boolean;
  files: FileType[];
  setFiles: (files: FileType[]) => void;
+  optimizationMode: string;
 }) => {
  const [loading, setLoading] = useState(false);
-  const fileInputRef = useRef<any>();
+  const fileInputRef = useRef<HTMLInputElement>(null);
+
+  const isSpeedMode = optimizationMode === 'speed';
+  const isDisabled = isSpeedMode;

  const handleChange = async (e: React.ChangeEvent<HTMLInputElement>) => {
+    if (isDisabled) return;
+
    setLoading(true);
    const data = new FormData();

@ -37,9 +42,18 @@ const Attach = ({
      'embeddingModelProvider',
    );
    const embeddingModel = localStorage.getItem('embeddingModel');
+    const chatModelProvider = localStorage.getItem('chatModelProvider');
+    const chatModel = localStorage.getItem('chatModel');
+    const ollamaContextWindow =
+      localStorage.getItem('ollamaContextWindow') || '2048';

    data.append('embedding_model_provider', embeddingModelProvider!);
    data.append('embedding_model', embeddingModel!);
+    data.append('chat_model_provider', chatModelProvider!);
+    data.append('chat_model', chatModel!);
+    if (chatModelProvider === 'ollama') {
+      data.append('ollama_context_window', ollamaContextWindow);
+    }

    const res = await fetch(`/api/uploads`, {
      method: 'POST',
@ -61,18 +75,31 @@ const Attach = ({
      </p>
    </div>
  ) : files.length > 0 ? (
+    <div className="relative group">
      <Popover className="relative w-full max-w-[15rem] md:max-w-md lg:max-w-lg">
        <PopoverButton
          type="button"
+          disabled={isDisabled}
          className={cn(
-          'flex flex-row items-center justify-between space-x-1 p-2 text-black/50 dark:text-white/50 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary active:scale-95 transition duration-200 hover:text-black dark:hover:text-white',
+            'flex flex-row items-center justify-between space-x-1 p-2 rounded-xl transition duration-200',
            files.length > 0 ? '-ml-2 lg:-ml-3' : '',
+            isDisabled
+              ? 'text-black/20 dark:text-white/20 cursor-not-allowed'
+              : 'text-black/50 dark:text-white/50 hover:bg-light-secondary dark:hover:bg-dark-secondary hover:text-black dark:hover:text-white',
          )}
        >
          {files.length > 1 && (
            <>
-            <File size={19} className="text-sky-400" />
-            <p className="text-sky-400 inline whitespace-nowrap text-xs font-medium">
+              <File
+                size={19}
+                className={isDisabled ? 'text-sky-900' : 'text-sky-400'}
+              />
+              <p
+                className={cn(
+                  'inline whitespace-nowrap text-xs font-medium',
+                  isDisabled ? 'text-sky-900' : 'text-sky-400',
+                )}
+              >
                {files.length} files
              </p>
            </>
@ -80,8 +107,16 @@ const Attach = ({

          {files.length === 1 && (
            <>
-            <File size={18} className="text-sky-400" />
-            <p className="text-sky-400 text-xs font-medium">
+              <File
+                size={18}
+                className={isDisabled ? 'text-sky-900' : 'text-sky-400'}
+              />
+              <p
+                className={cn(
+                  'text-xs font-medium',
+                  isDisabled ? 'text-sky-900' : 'text-sky-400',
+                )}
+              >
                {files[0].fileName.length > 10
                  ? files[0].fileName.replace(/\.\w+$/, '').substring(0, 3) +
                    '...' +
@ -109,8 +144,14 @@ const Attach = ({
                <div className="flex flex-row items-center space-x-4">
                  <button
                    type="button"
-                  onClick={() => fileInputRef.current.click()}
-                  className="flex flex-row items-center space-x-1 text-black/70 dark:text-white/70 hover:text-black hover:dark:text-white transition duration-200"
+                    onClick={() => !isDisabled && fileInputRef.current?.click()}
+                    disabled={isDisabled}
+                    className={cn(
+                      'flex flex-row items-center space-x-1 transition duration-200',
+                      isDisabled
+                        ? 'text-black/20 dark:text-white/20 cursor-not-allowed'
+                        : 'text-black/70 dark:text-white/70 hover:text-black hover:dark:text-white',
+                    )}
                  >
                    <input
                      type="file"
@ -119,16 +160,25 @@ const Attach = ({
                      accept=".pdf,.docx,.txt"
                      multiple
                      hidden
+                      disabled={isDisabled}
                    />
                    <Plus size={18} />
                    <p className="text-xs">Add</p>
                  </button>
                  <button
                    onClick={() => {
+                      if (!isDisabled) {
                        setFiles([]);
                        setFileIds([]);
+                      }
                    }}
-                  className="flex flex-row items-center space-x-1 text-black/70 dark:text-white/70 hover:text-black hover:dark:text-white transition duration-200"
+                    disabled={isDisabled}
+                    className={cn(
+                      'flex flex-row items-center space-x-1 transition duration-200',
+                      isDisabled
+                        ? 'text-black/20 dark:text-white/20 cursor-not-allowed'
+                        : 'text-black/70 dark:text-white/70 hover:text-black hover:dark:text-white',
+                    )}
                  >
                    <Trash size={14} />
                    <p className="text-xs">Clear</p>
@ -159,13 +209,26 @@ const Attach = ({
          </PopoverPanel>
        </Transition>
      </Popover>
+      {isSpeedMode && (
+        <div className="absolute bottom-full mb-2 left-1/2 transform -translate-x-1/2 opacity-0 group-hover:opacity-100 transition-opacity duration-200 pointer-events-none">
+          <div className="bg-black dark:bg-white text-white dark:text-black text-xs px-2 py-1 rounded whitespace-nowrap">
+            File attachments are disabled in Speed mode
+            <div className="absolute top-full left-1/2 transform -translate-x-1/2 border-4 border-transparent border-t-black dark:border-t-white"></div>
+          </div>
+        </div>
+      )}
+    </div>
  ) : (
+    <div className="relative group">
      <button
        type="button"
-      onClick={() => fileInputRef.current.click()}
+        onClick={() => !isDisabled && fileInputRef.current?.click()}
+        disabled={isDisabled}
        className={cn(
-        'flex flex-row items-center space-x-1 text-black/50 dark:text-white/50 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary transition duration-200 hover:text-black dark:hover:text-white',
-        showText ? '' : 'p-2',
+          'flex flex-row items-center space-x-1 rounded-xl transition duration-200 p-2',
+          isDisabled
+            ? 'text-black/20 dark:text-white/20 cursor-not-allowed'
+            : 'text-black/50 dark:text-white/50 hover:bg-light-secondary dark:hover:bg-dark-secondary hover:text-black dark:hover:text-white',
        )}
      >
        <input
@ -175,10 +238,19 @@ const Attach = ({
          accept=".pdf,.docx,.txt"
          multiple
          hidden
+          disabled={isDisabled}
        />
-      <CopyPlus size={showText ? 18 : undefined} />
-      {showText && <p className="text-xs font-medium pl-[1px]">Attach</p>}
+        <Paperclip size="18" />
      </button>
+      {isSpeedMode && (
+        <div className="absolute bottom-full mb-2 left-1/2 transform -translate-x-1/2 opacity-0 group-hover:opacity-100 transition-opacity duration-200 pointer-events-none">
+          <div className="bg-black dark:bg-white text-white dark:text-black text-xs px-2 py-1 rounded whitespace-nowrap">
+            File attachments are disabled in Speed mode
+            <div className="absolute top-full left-1/2 transform -translate-x-1/2 border-4 border-transparent border-t-black dark:border-t-white"></div>
+          </div>
+        </div>
+      )}
+    </div>
  );
 };

--- a/src/components/MessageInputActions/AttachSmall.tsx
+++ b/src/components/MessageInputActions/AttachSmall.tsx
@ -1,153 +0,0 @@
-import { cn } from '@/lib/utils';
-import {
-  Popover,
-  PopoverButton,
-  PopoverPanel,
-  Transition,
-} from '@headlessui/react';
-import { CopyPlus, File, LoaderCircle, Plus, Trash } from 'lucide-react';
-import { Fragment, useRef, useState } from 'react';
-import { File as FileType } from '../ChatWindow';
-
-const AttachSmall = ({
-  fileIds,
-  setFileIds,
-  files,
-  setFiles,
-}: {
-  fileIds: string[];
-  setFileIds: (fileIds: string[]) => void;
-  files: FileType[];
-  setFiles: (files: FileType[]) => void;
-}) => {
-  const [loading, setLoading] = useState(false);
-  const fileInputRef = useRef<any>();
-
-  const handleChange = async (e: React.ChangeEvent<HTMLInputElement>) => {
-    setLoading(true);
-    const data = new FormData();
-
-    for (let i = 0; i < e.target.files!.length; i++) {
-      data.append('files', e.target.files![i]);
-    }
-
-    const embeddingModelProvider = localStorage.getItem(
-      'embeddingModelProvider',
-    );
-    const embeddingModel = localStorage.getItem('embeddingModel');
-
-    data.append('embedding_model_provider', embeddingModelProvider!);
-    data.append('embedding_model', embeddingModel!);
-
-    const res = await fetch(`/api/uploads`, {
-      method: 'POST',
-      body: data,
-    });
-
-    const resData = await res.json();
-
-    setFiles([...files, ...resData.files]);
-    setFileIds([...fileIds, ...resData.files.map((file: any) => file.fileId)]);
-    setLoading(false);
-  };
-
-  return loading ? (
-    <div className="flex flex-row items-center justify-between space-x-1 p-1">
-      <LoaderCircle size={20} className="text-sky-400 animate-spin" />
-    </div>
-  ) : files.length > 0 ? (
-    <Popover className="max-w-[15rem] md:max-w-md lg:max-w-lg">
-      <PopoverButton
-        type="button"
-        className="flex flex-row items-center justify-between space-x-1 p-1 text-black/50 dark:text-white/50 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary active:scale-95 transition duration-200 hover:text-black dark:hover:text-white"
-      >
-        <File size={20} className="text-sky-400" />
-      </PopoverButton>
-      <Transition
-        as={Fragment}
-        enter="transition ease-out duration-150"
-        enterFrom="opacity-0 translate-y-1"
-        enterTo="opacity-100 translate-y-0"
-        leave="transition ease-in duration-150"
-        leaveFrom="opacity-100 translate-y-0"
-        leaveTo="opacity-0 translate-y-1"
-      >
-        <PopoverPanel className="absolute z-10 w-64 md:w-[350px] bottom-14 -ml-3">
-          <div className="bg-light-primary dark:bg-dark-primary border rounded-md border-light-200 dark:border-dark-200 w-full max-h-[200px] md:max-h-none overflow-y-auto flex flex-col">
-            <div className="flex flex-row items-center justify-between px-3 py-2">
-              <h4 className="text-black dark:text-white font-medium text-sm">
-                Attached files
-              </h4>
-              <div className="flex flex-row items-center space-x-4">
-                <button
-                  type="button"
-                  onClick={() => fileInputRef.current.click()}
-                  className="flex flex-row items-center space-x-1 text-black/70 dark:text-white/70 hover:text-black hover:dark:text-white transition duration-200"
-                >
-                  <input
-                    type="file"
-                    onChange={handleChange}
-                    ref={fileInputRef}
-                    accept=".pdf,.docx,.txt"
-                    multiple
-                    hidden
-                  />
-                  <Plus size={18} />
-                  <p className="text-xs">Add</p>
-                </button>
-                <button
-                  onClick={() => {
-                    setFiles([]);
-                    setFileIds([]);
-                  }}
-                  className="flex flex-row items-center space-x-1 text-black/70 dark:text-white/70 hover:text-black hover:dark:text-white transition duration-200"
-                >
-                  <Trash size={14} />
-                  <p className="text-xs">Clear</p>
-                </button>
-              </div>
-            </div>
-            <div className="h-[0.5px] mx-2 bg-white/10" />
-            <div className="flex flex-col items-center">
-              {files.map((file, i) => (
-                <div
-                  key={i}
-                  className="flex flex-row items-center justify-start w-full space-x-3 p-3"
-                >
-                  <div className="bg-dark-100 flex items-center justify-center w-10 h-10 rounded-md">
-                    <File size={16} className="text-white/70" />
-                  </div>
-                  <p className="text-black/70 dark:text-white/70 text-sm">
-                    {file.fileName.length > 25
-                      ? file.fileName.replace(/\.\w+$/, '').substring(0, 25) +
-                        '...' +
-                        file.fileExtension
-                      : file.fileName}
-                  </p>
-                </div>
-              ))}
-            </div>
-          </div>
-        </PopoverPanel>
-      </Transition>
-    </Popover>
-  ) : (
-    <button
-      type="button"
-      onClick={() => fileInputRef.current.click()}
-      className="flex flex-row items-center space-x-1 text-black/50 dark:text-white/50 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary transition duration-200 hover:text-black dark:hover:text-white p-1"
-    >
-      <input
-        type="file"
-        onChange={handleChange}
-        ref={fileInputRef}
-        accept=".pdf,.docx,.txt"
-        multiple
-        hidden
-      />
-      <CopyPlus size={20} />
-    </button>
-  );
-};
-
-export default AttachSmall;
--- a/src/components/MessageInputActions/Focus.tsx
+++ b/src/components/MessageInputActions/Focus.tsx
@ -1,57 +1,25 @@
-import {
-  BadgePercent,
-  ChevronDown,
-  Globe,
-  Pencil,
-  ScanEye,
-  SwatchBook,
-} from 'lucide-react';
+import { Globe, MessageCircle, Pencil } from 'lucide-react';
 import { cn } from '@/lib/utils';
-import {
-  Popover,
-  PopoverButton,
-  PopoverPanel,
-  Transition,
-} from '@headlessui/react';
-import { SiReddit, SiYoutube } from '@icons-pack/react-simple-icons';
-import { Fragment } from 'react';
+import { useState } from 'react';

 const focusModes = [
  {
    key: 'webSearch',
    title: 'All',
    description: 'Searches across all of the internet',
-    icon: <Globe size={20} />,
+    icon: <Globe size={20} className="text-[#24A0ED]" />,
  },
  {
-    key: 'academicSearch',
-    title: 'Academic',
-    description: 'Search in published academic papers',
-    icon: <SwatchBook size={20} />,
+    key: 'chat',
+    title: 'Chat',
+    description: 'Have a creative conversation',
+    icon: <MessageCircle size={16} className="text-[#10B981]" />,
  },
  {
-    key: 'writingAssistant',
-    title: 'Writing',
-    description: 'Chat without searching the web',
-    icon: <Pencil size={16} />,
-  },
-  {
-    key: 'wolframAlphaSearch',
-    title: 'Wolfram Alpha',
-    description: 'Computational knowledge engine',
-    icon: <BadgePercent size={20} />,
-  },
-  {
-    key: 'youtubeSearch',
-    title: 'Youtube',
-    description: 'Search and watch videos',
-    icon: <SiYoutube className="h-5 w-auto mr-0.5" />,
-  },
-  {
-    key: 'redditSearch',
-    title: 'Reddit',
-    description: 'Search for discussions and opinions',
-    icon: <SiReddit className="h-5 w-auto mr-0.5" />,
+    key: 'localResearch',
+    title: 'Local Research',
+    description: 'Research and interact with local files with citations',
+    icon: <Pencil size={16} className="text-[#8B5CF6]" />,
  },
 ];

@ -62,69 +30,136 @@ const Focus = ({
  focusMode: string;
  setFocusMode: (mode: string) => void;
 }) => {
+  const [showWebSearchTooltip, setShowWebSearchTooltip] = useState(false);
+  const [showChatTooltip, setShowChatTooltip] = useState(false);
+  const [showLocalResearchTooltip, setShowLocalResearchTooltip] =
+    useState(false);
+
+  const webSearchMode = focusModes.find((mode) => mode.key === 'webSearch');
+  const chatMode = focusModes.find((mode) => mode.key === 'chat');
+  const localResearchMode = focusModes.find(
+    (mode) => mode.key === 'localResearch',
+  );
+
  return (
-    <Popover className="relative w-full max-w-[15rem] md:max-w-md lg:max-w-lg mt-[6.5px]">
-      <PopoverButton
-        type="button"
-        className=" text-black/50 dark:text-white/50 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary active:scale-95 transition duration-200 hover:text-black dark:hover:text-white"
-      >
-        {focusMode !== 'webSearch' ? (
+    <div className="text-black/50 dark:text-white/50 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary transition duration-200 hover:text-black dark:hover:text-white">
      <div className="flex flex-row items-center space-x-1">
-            {focusModes.find((mode) => mode.key === focusMode)?.icon}
-            <p className="text-xs font-medium hidden lg:block">
-              {focusModes.find((mode) => mode.key === focusMode)?.title}
-            </p>
-            <ChevronDown size={20} className="-translate-x-1" />
-          </div>
-        ) : (
-          <div className="flex flex-row items-center space-x-1">
-            <ScanEye size={20} />
-            <p className="text-xs font-medium hidden lg:block">Focus</p>
-          </div>
-        )}
-      </PopoverButton>
-      <Transition
-        as={Fragment}
-        enter="transition ease-out duration-150"
-        enterFrom="opacity-0 translate-y-1"
-        enterTo="opacity-100 translate-y-0"
-        leave="transition ease-in duration-150"
-        leaveFrom="opacity-100 translate-y-0"
-        leaveTo="opacity-0 translate-y-1"
-      >
-        <PopoverPanel className="absolute z-10 w-64 md:w-[500px] left-0">
-          <div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-2 bg-light-primary dark:bg-dark-primary border rounded-lg border-light-200 dark:border-dark-200 w-full p-4 max-h-[200px] md:max-h-none overflow-y-auto">
-            {focusModes.map((mode, i) => (
-              <PopoverButton
-                onClick={() => setFocusMode(mode.key)}
-                key={i}
+        <div className="relative">
+          <div className="flex items-center border border-light-200 dark:border-dark-200 rounded-lg overflow-hidden">
+            {/* Web Search Mode Icon */}
+            <button
              className={cn(
-                  'p-2 rounded-lg flex flex-col items-start justify-start text-start space-y-2 duration-200 cursor-pointer transition',
-                  focusMode === mode.key
-                    ? 'bg-light-secondary dark:bg-dark-secondary'
-                    : 'hover:bg-light-secondary dark:hover:bg-dark-secondary',
+                'p-2 transition-all duration-200',
+                focusMode === 'webSearch'
+                  ? 'bg-[#24A0ED]/20 text-[#24A0ED] scale-105'
+                  : 'text-black/30 dark:text-white/30 hover:text-black/50 dark:hover:text-white/50 hover:bg-light-secondary/50 dark:hover:bg-dark-secondary/50',
              )}
+              onMouseEnter={() => setShowWebSearchTooltip(true)}
+              onMouseLeave={() => setShowWebSearchTooltip(false)}
+              onClick={(e) => {
+                e.stopPropagation();
+                setFocusMode('webSearch');
+              }}
            >
-                <div
+              <Globe size={18} />
+            </button>
+
+            {/* Divider */}
+            <div className="h-6 w-px bg-light-200 dark:bg-dark-200"></div>
+
+            {/* Chat Mode Icon */}
+            <button
              className={cn(
-                    'flex flex-row items-center space-x-1',
-                    focusMode === mode.key
-                      ? 'text-[#24A0ED]'
-                      : 'text-black dark:text-white',
+                'p-2 transition-all duration-200',
+                focusMode === 'chat'
+                  ? 'bg-[#10B981]/20 text-[#10B981] scale-105'
+                  : 'text-black/30 dark:text-white/30 hover:text-black/50 dark:hover:text-white/50 hover:bg-light-secondary/50 dark:hover:bg-dark-secondary/50',
              )}
+              onMouseEnter={() => setShowChatTooltip(true)}
+              onMouseLeave={() => setShowChatTooltip(false)}
+              onClick={(e) => {
+                e.stopPropagation();
+                setFocusMode('chat');
+              }}
            >
-                  {mode.icon}
-                  <p className="text-sm font-medium">{mode.title}</p>
+              <MessageCircle size={18} />
+            </button>
+
+            {/* Divider */}
+            <div className="h-6 w-px bg-light-200 dark:bg-dark-200"></div>
+
+            {/* Local Research Mode Icon */}
+            <button
+              className={cn(
+                'p-2 transition-all duration-200',
+                focusMode === 'localResearch'
+                  ? 'bg-[#8B5CF6]/20 text-[#8B5CF6] scale-105'
+                  : 'text-black/30 dark:text-white/30 hover:text-black/50 dark:hover:text-white/50 hover:bg-light-secondary/50 dark:hover:bg-dark-secondary/50',
+              )}
+              onMouseEnter={() => setShowLocalResearchTooltip(true)}
+              onMouseLeave={() => setShowLocalResearchTooltip(false)}
+              onClick={(e) => {
+                e.stopPropagation();
+                setFocusMode('localResearch');
+              }}
+            >
+              <Pencil size={18} />
+            </button>
          </div>
-                <p className="text-black/70 dark:text-white/70 text-xs">
-                  {mode.description}
+
+          {/* Web Search Mode Tooltip */}
+          {showWebSearchTooltip && (
+            <div className="absolute z-20 bottom-[100%] mb-2 left-0 animate-in fade-in-0 duration-150">
+              <div className="bg-light-primary dark:bg-dark-primary border rounded-lg border-light-200 dark:border-dark-200 p-4 w-80 shadow-lg">
+                <div className="flex items-center space-x-2 mb-2">
+                  <Globe size={16} className="text-[#24A0ED]" />
+                  <h3 className="font-medium text-sm text-black dark:text-white text-left">
+                    {webSearchMode?.title}
+                  </h3>
+                </div>
+                <p className="text-sm text-black/70 dark:text-white/70 leading-relaxed text-left">
+                  {webSearchMode?.description}
                </p>
-              </PopoverButton>
-            ))}
              </div>
-        </PopoverPanel>
-      </Transition>
-    </Popover>
+            </div>
+          )}
+
+          {/* Chat Mode Tooltip */}
+          {showChatTooltip && (
+            <div className="absolute z-20 bottom-[100%] mb-2 left-0 transform animate-in fade-in-0 duration-150">
+              <div className="bg-light-primary dark:bg-dark-primary border rounded-lg border-light-200 dark:border-dark-200 p-4 w-80 shadow-lg">
+                <div className="flex items-center space-x-2 mb-2">
+                  <MessageCircle size={16} className="text-[#10B981]" />
+                  <h3 className="font-medium text-sm text-black dark:text-white text-left">
+                    {chatMode?.title}
+                  </h3>
+                </div>
+                <p className="text-sm text-black/70 dark:text-white/70 leading-relaxed text-left">
+                  {chatMode?.description}
+                </p>
+              </div>
+            </div>
+          )}
+
+          {/* Local Research Mode Tooltip */}
+          {showLocalResearchTooltip && (
+            <div className="absolute z-20 bottom-[100%] mb-2 left-0 animate-in fade-in-0 duration-150">
+              <div className="bg-light-primary dark:bg-dark-primary border rounded-lg border-light-200 dark:border-dark-200 p-4 w-80 shadow-lg">
+                <div className="flex items-center space-x-2 mb-2">
+                  <Pencil size={16} className="text-[#8B5CF6]" />
+                  <h3 className="font-medium text-sm text-black dark:text-white text-left">
+                    {localResearchMode?.title}
+                  </h3>
+                </div>
+                <p className="text-sm text-black/70 dark:text-white/70 leading-relaxed text-left">
+                  {localResearchMode?.description}
+                </p>
+              </div>
+            </div>
+          )}
+        </div>
+      </div>
+    </div>
  );
 };

--- a/src/components/MessageInputActions/ModelSelector.tsx
+++ b/src/components/MessageInputActions/ModelSelector.tsx
@ -0,0 +1,323 @@
+import { useEffect, useState } from 'react';
+import { Cpu, ChevronDown, ChevronRight } from 'lucide-react';
+import { cn } from '@/lib/utils';
+import {
+  Popover,
+  PopoverButton,
+  PopoverPanel,
+  Transition,
+} from '@headlessui/react';
+import { Fragment } from 'react';
+
+interface ModelOption {
+  provider: string;
+  model: string;
+  displayName: string;
+}
+
+interface ProviderModelMap {
+  [provider: string]: {
+    displayName: string;
+    models: ModelOption[];
+  };
+}
+
+const ModelSelector = ({
+  selectedModel,
+  setSelectedModel,
+  truncateModelName = true,
+  showModelName = true,
+}: {
+  selectedModel: { provider: string; model: string } | null;
+  setSelectedModel: (model: { provider: string; model: string }) => void;
+  truncateModelName?: boolean;
+  showModelName?: boolean;
+}) => {
+  const [providerModels, setProviderModels] = useState<ProviderModelMap>({});
+  const [providersList, setProvidersList] = useState<string[]>([]);
+  const [loading, setLoading] = useState(true);
+  const [selectedModelDisplay, setSelectedModelDisplay] = useState<string>('');
+  const [selectedProviderDisplay, setSelectedProviderDisplay] =
+    useState<string>('');
+  const [expandedProviders, setExpandedProviders] = useState<
+    Record<string, boolean>
+  >({});
+
+  useEffect(() => {
+    const fetchModels = async () => {
+      try {
+        const response = await fetch('/api/models', {
+          headers: {
+            'Content-Type': 'application/json',
+          },
+        });
+
+        if (!response.ok) {
+          throw new Error(`Failed to fetch models: ${response.status}`);
+        }
+
+        const data = await response.json();
+        const providersData: ProviderModelMap = {};
+
+        // Organize models by provider
+        Object.entries(data.chatModelProviders).forEach(
+          ([provider, models]: [string, any]) => {
+            const providerDisplayName =
+              provider.charAt(0).toUpperCase() + provider.slice(1);
+            providersData[provider] = {
+              displayName: providerDisplayName,
+              models: [],
+            };
+
+            Object.entries(models).forEach(
+              ([modelKey, modelData]: [string, any]) => {
+                providersData[provider].models.push({
+                  provider,
+                  model: modelKey,
+                  displayName: modelData.displayName || modelKey,
+                });
+              },
+            );
+          },
+        );
+
+        // Filter out providers with no models
+        Object.keys(providersData).forEach((provider) => {
+          if (providersData[provider].models.length === 0) {
+            delete providersData[provider];
+          }
+        });
+
+        // Sort providers by name (only those that have models)
+        const sortedProviders = Object.keys(providersData).sort();
+        setProvidersList(sortedProviders);
+
+        // Initialize expanded state for all providers
+        const initialExpandedState: Record<string, boolean> = {};
+        sortedProviders.forEach((provider) => {
+          initialExpandedState[provider] = selectedModel?.provider === provider;
+        });
+
+        // Expand the first provider if none is selected
+        if (sortedProviders.length > 0 && !selectedModel) {
+          initialExpandedState[sortedProviders[0]] = true;
+        }
+
+        setExpandedProviders(initialExpandedState);
+        setProviderModels(providersData);
+
+        // Find the current model in our options to display its name
+        if (selectedModel) {
+          const provider = providersData[selectedModel.provider];
+          if (provider) {
+            const currentModel = provider.models.find(
+              (option) => option.model === selectedModel.model,
+            );
+
+            if (currentModel) {
+              setSelectedModelDisplay(currentModel.displayName);
+              setSelectedProviderDisplay(provider.displayName);
+            }
+          } else {
+            setSelectedModelDisplay('');
+            setSelectedProviderDisplay('');
+          }
+        } else {
+          setSelectedModelDisplay('');
+          setSelectedProviderDisplay('');
+        }
+
+        setLoading(false);
+      } catch (error) {
+        console.error('Error fetching models:', error);
+        setLoading(false);
+      }
+    };
+
+    fetchModels();
+  }, [selectedModel]);
+
+  const toggleProviderExpanded = (provider: string) => {
+    setExpandedProviders((prev) => ({
+      ...prev,
+      [provider]: !prev[provider],
+    }));
+  };
+
+  const handleSelectModel = (option: ModelOption) => {
+    setSelectedModel({
+      provider: option.provider,
+      model: option.model,
+    });
+
+    setSelectedModelDisplay(option.displayName);
+    setSelectedProviderDisplay(
+      providerModels[option.provider]?.displayName || option.provider,
+    );
+  };
+
+  const getDisplayText = () => {
+    if (loading) return 'Loading...';
+    if (!selectedModelDisplay) return 'Select model';
+
+    return `${selectedModelDisplay} (${selectedProviderDisplay})`;
+  };
+
+  return (
+    <Popover className="relative">
+      {({ open }) => (
+        <>
+          <div className="relative">
+            <PopoverButton
+              type="button"
+              className="p-2 group flex text-black/50 dark:text-white/50 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary active:scale-95 transition duration-200 hover:text-black dark:hover:text-white"
+            >
+              <Cpu size={18} />
+              {showModelName && (
+                <span
+                  className={cn(
+                    'ml-2 text-xs font-medium overflow-hidden text-ellipsis whitespace-nowrap',
+                    {
+                      'max-w-44': truncateModelName,
+                    },
+                  )}
+                >
+                  {getDisplayText()}
+                </span>
+              )}
+              <ChevronDown
+                size={16}
+                className={cn(
+                  'transition-transform',
+                  open ? 'rotate-180' : 'rotate-0',
+                )}
+              />
+            </PopoverButton>
+          </div>
+
+          <Transition
+            as={Fragment}
+            enter="transition ease-out duration-200"
+            enterFrom="opacity-0 translate-y-1"
+            enterTo="opacity-100 translate-y-0"
+            leave="transition ease-in duration-150"
+            leaveFrom="opacity-100 translate-y-0"
+            leaveTo="opacity-0 translate-y-1"
+          >
+            <PopoverPanel className="absolute z-10 w-72 transform bottom-full mb-2">
+              <div className="overflow-hidden rounded-lg shadow-lg ring-1 ring-black/5 dark:ring-white/5 bg-white dark:bg-dark-secondary divide-y divide-light-200 dark:divide-dark-200">
+                <div className="px-4 py-3">
+                  <h3 className="text-sm font-medium text-black/90 dark:text-white/90">
+                    Select Model
+                  </h3>
+                  <p className="text-xs text-black/60 dark:text-white/60 mt-1">
+                    Choose a provider and model for your conversation
+                  </p>
+                </div>
+                <div className="max-h-72 overflow-y-auto">
+                  {loading ? (
+                    <div className="px-4 py-3 text-sm text-black/70 dark:text-white/70">
+                      Loading available models...
+                    </div>
+                  ) : providersList.length === 0 ? (
+                    <div className="px-4 py-3 text-sm text-black/70 dark:text-white/70">
+                      No models available
+                    </div>
+                  ) : (
+                    <div className="py-1">
+                      {providersList.map((providerKey) => {
+                        const provider = providerModels[providerKey];
+                        const isExpanded = expandedProviders[providerKey];
+
+                        return (
+                          <div
+                            key={providerKey}
+                            className="border-t border-light-200 dark:border-dark-200 first:border-t-0"
+                          >
+                            {/* Provider header */}
+                            <button
+                              className={cn(
+                                'w-full flex items-center justify-between px-4 py-2 text-sm text-left',
+                                'hover:bg-light-100 dark:hover:bg-dark-100',
+                                selectedModel?.provider === providerKey
+                                  ? 'bg-light-50 dark:bg-dark-50'
+                                  : '',
+                              )}
+                              onClick={() =>
+                                toggleProviderExpanded(providerKey)
+                              }
+                            >
+                              <div className="font-medium flex items-center">
+                                <Cpu
+                                  size={14}
+                                  className="mr-2 text-black/70 dark:text-white/70"
+                                />
+                                {provider.displayName}
+                                {selectedModel?.provider === providerKey && (
+                                  <span className="ml-2 text-xs text-[#24A0ED]">
+                                    (active)
+                                  </span>
+                                )}
+                              </div>
+                              <ChevronRight
+                                size={14}
+                                className={cn(
+                                  'transition-transform',
+                                  isExpanded ? 'rotate-90' : '',
+                                )}
+                              />
+                            </button>
+
+                            {/* Models list */}
+                            {isExpanded && (
+                              <div className="pl-6">
+                                {provider.models.map((modelOption) => (
+                                  <PopoverButton
+                                    key={`${modelOption.provider}-${modelOption.model}`}
+                                    className={cn(
+                                      'w-full text-left px-4 py-2 text-sm flex items-center',
+                                      selectedModel?.provider ===
+                                        modelOption.provider &&
+                                        selectedModel?.model ===
+                                          modelOption.model
+                                        ? 'bg-light-100 dark:bg-dark-100 text-black dark:text-white'
+                                        : 'text-black/70 dark:text-white/70 hover:bg-light-100 dark:hover:bg-dark-100',
+                                    )}
+                                    onClick={() =>
+                                      handleSelectModel(modelOption)
+                                    }
+                                  >
+                                    <div className="flex flex-col flex-1">
+                                      <span className="font-medium">
+                                        {modelOption.displayName}
+                                      </span>
+                                    </div>
+                                    {/* Active indicator */}
+                                    {selectedModel?.provider ===
+                                      modelOption.provider &&
+                                      selectedModel?.model ===
+                                        modelOption.model && (
+                                        <div className="ml-auto bg-[#24A0ED] text-white text-xs px-1.5 py-0.5 rounded">
+                                          Active
+                                        </div>
+                                      )}
+                                  </PopoverButton>
+                                ))}
+                              </div>
+                            )}
+                          </div>
+                        );
+                      })}
+                    </div>
+                  )}
+                </div>
+              </div>
+            </PopoverPanel>
+          </Transition>
+        </>
+      )}
+    </Popover>
+  );
+};
+
+export default ModelSelector;
--- a/src/components/MessageInputActions/Optimization.tsx
+++ b/src/components/MessageInputActions/Optimization.tsx
@ -1,103 +1,130 @@
-import { ChevronDown, Sliders, Star, Zap } from 'lucide-react';
+import { Zap, Bot } from 'lucide-react';
 import { cn } from '@/lib/utils';
-import {
-  Popover,
-  PopoverButton,
-  PopoverPanel,
-  Transition,
-} from '@headlessui/react';
-import { Fragment } from 'react';
+import { useState } from 'react';

 const OptimizationModes = [
  {
    key: 'speed',
    title: 'Speed',
-    description: 'Prioritize speed and get the quickest possible answer.',
+    description:
+      'Prioritize speed and get the quickest possible answer. Uses only web search results - attached files will not be processed.',
    icon: <Zap size={20} className="text-[#FF9800]" />,
  },
  {
-    key: 'balanced',
-    title: 'Balanced',
-    description: 'Find the right balance between speed and accuracy',
-    icon: <Sliders size={20} className="text-[#4CAF50]" />,
-  },
-  {
-    key: 'quality',
-    title: 'Quality (Soon)',
-    description: 'Get the most thorough and accurate answer',
-    icon: (
-      <Star
-        size={16}
-        className="text-[#2196F3] dark:text-[#BBDEFB] fill-[#BBDEFB] dark:fill-[#2196F3]"
-      />
-    ),
+    key: 'agent',
+    title: 'Agent (Experimental)',
+    description:
+      'Use an agentic workflow to answer complex multi-part questions. This mode may take longer and is experimental. It uses large prompts and may not work with all models. Best with at least a 8b model that supports 32k context or more.',
+    icon: <Bot size={20} className="text-[#9C27B0]" />,
  },
 ];

 const Optimization = ({
  optimizationMode,
  setOptimizationMode,
+  showTitle = false,
 }: {
  optimizationMode: string;
  setOptimizationMode: (mode: string) => void;
+  showTitle?: boolean;
 }) => {
+  const currentMode = OptimizationModes.find(
+    (mode) => mode.key === optimizationMode,
+  );
+  const isAgentMode = optimizationMode === 'agent';
+
+  const [showSpeedTooltip, setShowSpeedTooltip] = useState(false);
+  const [showAgentTooltip, setShowAgentTooltip] = useState(false);
+
+  const handleToggle = () => {
+    setOptimizationMode(isAgentMode ? 'speed' : 'agent');
+  };
+
+  const speedMode = OptimizationModes.find((mode) => mode.key === 'speed');
+  const agentMode = OptimizationModes.find((mode) => mode.key === 'agent');
+
  return (
-    <Popover className="relative w-full max-w-[15rem] md:max-w-md lg:max-w-lg">
-      <PopoverButton
+    <button
      type="button"
-        className="p-2 text-black/50 dark:text-white/50 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary active:scale-95 transition duration-200 hover:text-black dark:hover:text-white"
+      onClick={handleToggle}
+      className="text-black/50 dark:text-white/50 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary active:scale-95 transition duration-200 hover:text-black dark:hover:text-white"
    >
      <div className="flex flex-row items-center space-x-1">
-          {
-            OptimizationModes.find((mode) => mode.key === optimizationMode)
-              ?.icon
-          }
-          <p className="text-xs font-medium">
-            {
-              OptimizationModes.find((mode) => mode.key === optimizationMode)
-                ?.title
-            }
-          </p>
-          <ChevronDown size={20} />
-        </div>
-      </PopoverButton>
-      <Transition
-        as={Fragment}
-        enter="transition ease-out duration-150"
-        enterFrom="opacity-0 translate-y-1"
-        enterTo="opacity-100 translate-y-0"
-        leave="transition ease-in duration-150"
-        leaveFrom="opacity-100 translate-y-0"
-        leaveTo="opacity-0 translate-y-1"
-      >
-        <PopoverPanel className="absolute z-10 w-64 md:w-[250px] right-0">
-          <div className="flex flex-col gap-2 bg-light-primary dark:bg-dark-primary border rounded-lg border-light-200 dark:border-dark-200 w-full p-4 max-h-[200px] md:max-h-none overflow-y-auto">
-            {OptimizationModes.map((mode, i) => (
-              <PopoverButton
-                onClick={() => setOptimizationMode(mode.key)}
-                key={i}
-                disabled={mode.key === 'quality'}
+        <div className="relative">
+          <div className="flex items-center border border-light-200 dark:border-dark-200 rounded-lg overflow-hidden">
+            {/* Speed Mode Icon */}
+            <div
              className={cn(
-                  'p-2 rounded-lg flex flex-col items-start justify-start text-start space-y-1 duration-200 cursor-pointer transition',
-                  optimizationMode === mode.key
-                    ? 'bg-light-secondary dark:bg-dark-secondary'
-                    : 'hover:bg-light-secondary dark:hover:bg-dark-secondary',
-                  mode.key === 'quality' && 'opacity-50 cursor-not-allowed',
+                'p-2 transition-all duration-200',
+                !isAgentMode
+                  ? 'bg-[#FF9800]/20 text-[#FF9800] scale-105'
+                  : 'text-black/30 dark:text-white/30 hover:text-black/50 dark:hover:text-white/50 hover:bg-light-secondary/50 dark:hover:bg-dark-secondary/50',
              )}
+              onMouseEnter={() => setShowSpeedTooltip(true)}
+              onMouseLeave={() => setShowSpeedTooltip(false)}
            >
-                <div className="flex flex-row items-center space-x-1 text-black dark:text-white">
-                  {mode.icon}
-                  <p className="text-sm font-medium">{mode.title}</p>
+              <Zap size={18} />
            </div>
-                <p className="text-black/70 dark:text-white/70 text-xs">
-                  {mode.description}
+
+            {/* Divider */}
+            <div className="h-6 w-px bg-light-200 dark:bg-dark-200"></div>
+
+            {/* Agent Mode Icon */}
+            <div
+              className={cn(
+                'p-2 transition-all duration-200',
+                isAgentMode
+                  ? 'bg-[#9C27B0]/20 text-[#9C27B0] scale-105'
+                  : 'text-black/30 dark:text-white/30 hover:text-black/50 dark:hover:text-white/50 hover:bg-light-secondary/50 dark:hover:bg-dark-secondary/50',
+              )}
+              onMouseEnter={() => setShowAgentTooltip(true)}
+              onMouseLeave={() => setShowAgentTooltip(false)}
+            >
+              <Bot size={18} />
+            </div>
+          </div>
+
+          {/* Speed Mode Tooltip */}
+          {showSpeedTooltip && (
+            <div className="absolute z-20 bottom-[100%] mb-2 right-0 animate-in fade-in-0 duration-150">
+              <div className="bg-light-primary dark:bg-dark-primary border rounded-lg border-light-200 dark:border-dark-200 p-4 w-80 shadow-lg">
+                <div className="flex items-center space-x-2 mb-2">
+                  <Zap size={16} className="text-[#FF9800]" />
+                  <h3 className="font-medium text-sm text-black dark:text-white text-left">
+                    {speedMode?.title}
+                  </h3>
+                </div>
+                <p className="text-sm text-black/70 dark:text-white/70 leading-relaxed text-left">
+                  {speedMode?.description}
                </p>
-              </PopoverButton>
-            ))}
              </div>
-        </PopoverPanel>
-      </Transition>
-    </Popover>
+            </div>
+          )}
+
+          {/* Agent Mode Tooltip */}
+          {showAgentTooltip && (
+            <div className="absolute z-20 bottom-[100%] mb-2 right-0 animate-in fade-in-0 duration-150">
+              <div className="bg-light-primary dark:bg-dark-primary border rounded-lg border-light-200 dark:border-dark-200 p-4 w-80 shadow-lg">
+                <div className="flex items-center space-x-2 mb-2">
+                  <Bot size={16} className="text-[#9C27B0]" />
+                  <h3 className="font-medium text-sm text-black dark:text-white text-left">
+                    {agentMode?.title}
+                  </h3>
+                </div>
+                <p className="text-sm text-black/70 dark:text-white/70 leading-relaxed text-left">
+                  {agentMode?.description}
+                </p>
+              </div>
+            </div>
+          )}
+        </div>{' '}
+        {showTitle && (
+          <p className="text-xs font-medium ml-1">
+            {currentMode?.title || 'Speed'}
+          </p>
+        )}
+      </div>
+    </button>
  );
 };

--- a/src/components/MessageInputActions/SystemPromptSelector.tsx
+++ b/src/components/MessageInputActions/SystemPromptSelector.tsx
@ -0,0 +1,226 @@
+import {
+  BookUser,
+  ChevronDown,
+  CheckSquare,
+  Square,
+  Settings,
+  User,
+  Loader2,
+} from 'lucide-react';
+import { cn } from '@/lib/utils';
+import {
+  Popover,
+  PopoverButton,
+  PopoverPanel,
+  Transition,
+} from '@headlessui/react';
+import { Fragment, useEffect, useState } from 'react';
+
+interface SystemPrompt {
+  id: string;
+  name: string;
+  type: 'system' | 'persona';
+}
+
+interface SystemPromptSelectorProps {
+  selectedPromptIds: string[];
+  onSelectedPromptIdsChange: (ids: string[]) => void;
+}
+
+const SystemPromptSelector = ({
+  selectedPromptIds,
+  onSelectedPromptIdsChange,
+}: SystemPromptSelectorProps) => {
+  const [availablePrompts, setAvailablePrompts] = useState<SystemPrompt[]>([]);
+  const [isOpen, setIsOpen] = useState(false);
+  const [isLoading, setIsLoading] = useState(true);
+
+  useEffect(() => {
+    if (!isOpen) return; // Only fetch when popover is open or about to open
+    const fetchPrompts = async () => {
+      try {
+        setIsLoading(true);
+        const response = await fetch('/api/system-prompts');
+        if (response.ok) {
+          const prompts = await response.json();
+          setAvailablePrompts(prompts);
+
+          // Check if any currently selected prompt IDs are not in the API response
+          const availablePromptIds = prompts.map(
+            (prompt: SystemPrompt) => prompt.id,
+          );
+          const validSelectedIds = selectedPromptIds.filter((id) =>
+            availablePromptIds.includes(id),
+          );
+
+          // If some selected IDs are no longer available, update the selection
+          if (validSelectedIds.length !== selectedPromptIds.length) {
+            onSelectedPromptIdsChange(validSelectedIds);
+          }
+        } else {
+          console.error('Failed to load system prompts.');
+        }
+      } catch (error) {
+        console.error('Error loading system prompts.');
+        console.error(error);
+      } finally {
+        setIsLoading(false);
+      }
+    };
+    fetchPrompts();
+  }, [isOpen, selectedPromptIds, onSelectedPromptIdsChange]);
+
+  const handleTogglePrompt = (promptId: string) => {
+    const newSelectedIds = selectedPromptIds.includes(promptId)
+      ? selectedPromptIds.filter((id) => id !== promptId)
+      : [...selectedPromptIds, promptId];
+    onSelectedPromptIdsChange(newSelectedIds);
+  };
+
+  const selectedCount = selectedPromptIds.length;
+
+  return (
+    <Popover className="relative">
+      {({ open }) => {
+        if (open && !isOpen) setIsOpen(true);
+        if (!open && isOpen) setIsOpen(false);
+        return (
+          <>
+            <PopoverButton
+              className={cn(
+                'flex items-center gap-1 rounded-lg text-sm transition-colors duration-150 ease-in-out focus:outline-none focus-visible:ring-2 focus-visible:ring-blue-500',
+                selectedCount > 0
+                  ? 'text-[#24A0ED] hover:text-blue-200'
+                  : 'text-black/60 hover:text-black/30 dark:text-white/60 dark:hover:*:text-white/30',
+              )}
+              title="Select Prompts"
+            >
+              <BookUser size={18} />
+              {selectedCount > 0 ? <span> {selectedCount} </span> : null}
+              <ChevronDown size={16} className="opacity-60" />
+            </PopoverButton>
+            <Transition
+              as={Fragment}
+              enter="transition ease-out duration-200"
+              enterFrom="opacity-0 translate-y-1"
+              enterTo="opacity-100 translate-y-0"
+              leave="transition ease-in duration-150"
+              leaveFrom="opacity-100 translate-y-0"
+              leaveTo="opacity-0 translate-y-1"
+            >
+              <PopoverPanel className="absolute z-20 w-72 transform bottom-full mb-2">
+                <div className="overflow-hidden rounded-lg shadow-lg ring-1 ring-black/5 dark:ring-white/5 bg-white dark:bg-dark-secondary">
+                  <div className="px-4 py-3 border-b border-light-200 dark:border-dark-200">
+                    <h3 className="text-sm font-medium text-black/90 dark:text-white/90">
+                      Select Prompts
+                    </h3>
+                    <p className="text-xs text-black/60 dark:text-white/60 mt-0.5">
+                      Choose instructions to guide the AI.
+                    </p>
+                  </div>
+                  {isLoading ? (
+                    <div className="px-4 py-3">
+                      <Loader2 className="animate-spin text-black/70 dark:text-white/70" />
+                    </div>
+                  ) : (
+                    <div className="max-h-60 overflow-y-auto p-1.5 space-y-3">
+                      {availablePrompts.length === 0 && (
+                        <p className="text-xs text-black/50 dark:text-white/50 px-2.5 py-2 text-center">
+                          No prompts configured. <br /> Go to{' '}
+                          <a className="text-blue-500" href="/settings">
+                            settings
+                          </a>{' '}
+                          to add some.
+                        </p>
+                      )}
+
+                      {availablePrompts.filter((p) => p.type === 'system')
+                        .length > 0 && (
+                        <div>
+                          <div className="flex items-center gap-1.5 px-2.5 py-1.5 text-xs font-medium text-black/70 dark:text-white/70">
+                            <Settings size={14} />
+                            <span>System Prompts</span>
+                          </div>
+                          <div className="space-y-0.5">
+                            {availablePrompts
+                              .filter((p) => p.type === 'system')
+                              .map((prompt) => (
+                                <div
+                                  key={prompt.id}
+                                  onClick={() => handleTogglePrompt(prompt.id)}
+                                  className="flex items-center gap-2.5 p-2.5 rounded-md hover:bg-light-100 dark:hover:bg-dark-100 cursor-pointer"
+                                >
+                                  {selectedPromptIds.includes(prompt.id) ? (
+                                    <CheckSquare
+                                      size={18}
+                                      className="text-[#24A0ED] flex-shrink-0"
+                                    />
+                                  ) : (
+                                    <Square
+                                      size={18}
+                                      className="text-black/40 dark:text-white/40 flex-shrink-0"
+                                    />
+                                  )}
+                                  <span
+                                    className="text-sm text-black/80 dark:text-white/80 truncate"
+                                    title={prompt.name}
+                                  >
+                                    {prompt.name}
+                                  </span>
+                                </div>
+                              ))}
+                          </div>
+                        </div>
+                      )}
+
+                      {availablePrompts.filter((p) => p.type === 'persona')
+                        .length > 0 && (
+                        <div>
+                          <div className="flex items-center gap-1.5 px-2.5 py-1.5 text-xs font-medium text-black/70 dark:text-white/70">
+                            <User size={14} />
+                            <span>Persona Prompts</span>
+                          </div>
+                          <div className="space-y-0.5">
+                            {availablePrompts
+                              .filter((p) => p.type === 'persona')
+                              .map((prompt) => (
+                                <div
+                                  key={prompt.id}
+                                  onClick={() => handleTogglePrompt(prompt.id)}
+                                  className="flex items-center gap-2.5 p-2.5 rounded-md hover:bg-light-100 dark:hover:bg-dark-100 cursor-pointer"
+                                >
+                                  {selectedPromptIds.includes(prompt.id) ? (
+                                    <CheckSquare
+                                      size={18}
+                                      className="text-[#24A0ED] flex-shrink-0"
+                                    />
+                                  ) : (
+                                    <Square
+                                      size={18}
+                                      className="text-black/40 dark:text-white/40 flex-shrink-0"
+                                    />
+                                  )}
+                                  <span
+                                    className="text-sm text-black/80 dark:text-white/80 truncate"
+                                    title={prompt.name}
+                                  >
+                                    {prompt.name}
+                                  </span>
+                                </div>
+                              ))}
+                          </div>
+                        </div>
+                      )}
+                    </div>
+                  )}
+                </div>
+              </PopoverPanel>
+            </Transition>
+          </>
+        );
+      }}
+    </Popover>
+  );
+};
+
+export default SystemPromptSelector;
--- a/src/components/MessageSources.tsx
+++ b/src/components/MessageSources.tsx
@ -1,31 +1,11 @@
 /* eslint-disable @next/next/no-img-element */
-import {
-  Dialog,
-  DialogPanel,
-  DialogTitle,
-  Transition,
-  TransitionChild,
-} from '@headlessui/react';
 import { Document } from '@langchain/core/documents';
-import { File } from 'lucide-react';
-import { Fragment, useState } from 'react';
+import { File, Zap, Microscope } from 'lucide-react';

 const MessageSources = ({ sources }: { sources: Document[] }) => {
-  const [isDialogOpen, setIsDialogOpen] = useState(false);
-
-  const closeModal = () => {
-    setIsDialogOpen(false);
-    document.body.classList.remove('overflow-hidden-scrollable');
-  };
-
-  const openModal = () => {
-    setIsDialogOpen(true);
-    document.body.classList.add('overflow-hidden-scrollable');
-  };
-
  return (
    <div className="grid grid-cols-2 lg:grid-cols-4 gap-2">
-      {sources.slice(0, 3).map((source, i) => (
+      {sources.map((source, i) => (
        <a
          className="bg-light-100 hover:bg-light-200 dark:bg-dark-100 dark:hover:bg-dark-200 transition duration-200 rounded-lg p-3 flex flex-col space-y-2 font-medium"
          key={i}
@ -57,105 +37,27 @@ const MessageSources = ({ sources }: { sources: Document[] }) => {
            <div className="flex flex-row items-center space-x-1 text-black/50 dark:text-white/50 text-xs">
              <div className="bg-black/50 dark:bg-white/50 h-[4px] w-[4px] rounded-full" />
              <span>{i + 1}</span>
+              {/* Processing type indicator */}
+              {source.metadata.processingType === 'preview-only' && (
+                <span title="Partial content analyzed" className="inline-flex">
+                  <Zap
+                    size={14}
+                    className="text-black/40 dark:text-white/40 ml-1"
+                  />
+                </span>
+              )}
+              {source.metadata.processingType === 'full-content' && (
+                <span title="Full content analyzed" className="inline-flex">
+                  <Microscope
+                    size={14}
+                    className="text-black/40 dark:text-white/40 ml-1"
+                  />
+                </span>
+              )}
            </div>
          </div>
        </a>
      ))}
-      {sources.length > 3 && (
-        <button
-          onClick={openModal}
-          className="bg-light-100 hover:bg-light-200 dark:bg-dark-100 dark:hover:bg-dark-200 transition duration-200 rounded-lg p-3 flex flex-col space-y-2 font-medium"
-        >
-          <div className="flex flex-row items-center space-x-1">
-            {sources.slice(3, 6).map((source, i) => {
-              return source.metadata.url === 'File' ? (
-                <div
-                  key={i}
-                  className="bg-dark-200 hover:bg-dark-100 transition duration-200 flex items-center justify-center w-6 h-6 rounded-full"
-                >
-                  <File size={12} className="text-white/70" />
-                </div>
-              ) : (
-                <img
-                  key={i}
-                  src={`https://s2.googleusercontent.com/s2/favicons?domain_url=${source.metadata.url}`}
-                  width={16}
-                  height={16}
-                  alt="favicon"
-                  className="rounded-lg h-4 w-4"
-                />
-              );
-            })}
-          </div>
-          <p className="text-xs text-black/50 dark:text-white/50">
-            View {sources.length - 3} more
-          </p>
-        </button>
-      )}
-      <Transition appear show={isDialogOpen} as={Fragment}>
-        <Dialog as="div" className="relative z-50" onClose={closeModal}>
-          <div className="fixed inset-0 overflow-y-auto">
-            <div className="flex min-h-full items-center justify-center p-4 text-center">
-              <TransitionChild
-                as={Fragment}
-                enter="ease-out duration-200"
-                enterFrom="opacity-0 scale-95"
-                enterTo="opacity-100 scale-100"
-                leave="ease-in duration-100"
-                leaveFrom="opacity-100 scale-200"
-                leaveTo="opacity-0 scale-95"
-              >
-                <DialogPanel className="w-full max-w-md transform rounded-2xl bg-light-secondary dark:bg-dark-secondary border border-light-200 dark:border-dark-200 p-6 text-left align-middle shadow-xl transition-all">
-                  <DialogTitle className="text-lg font-medium leading-6 dark:text-white">
-                    Sources
-                  </DialogTitle>
-                  <div className="grid grid-cols-2 gap-2 overflow-auto max-h-[300px] mt-2 pr-2">
-                    {sources.map((source, i) => (
-                      <a
-                        className="bg-light-secondary hover:bg-light-200 dark:bg-dark-secondary dark:hover:bg-dark-200 border border-light-200 dark:border-dark-200 transition duration-200 rounded-lg p-3 flex flex-col space-y-2 font-medium"
-                        key={i}
-                        href={source.metadata.url}
-                        target="_blank"
-                      >
-                        <p className="dark:text-white text-xs overflow-hidden whitespace-nowrap text-ellipsis">
-                          {source.metadata.title}
-                        </p>
-                        <div className="flex flex-row items-center justify-between">
-                          <div className="flex flex-row items-center space-x-1">
-                            {source.metadata.url === 'File' ? (
-                              <div className="bg-dark-200 hover:bg-dark-100 transition duration-200 flex items-center justify-center w-6 h-6 rounded-full">
-                                <File size={12} className="text-white/70" />
-                              </div>
-                            ) : (
-                              <img
-                                src={`https://s2.googleusercontent.com/s2/favicons?domain_url=${source.metadata.url}`}
-                                width={16}
-                                height={16}
-                                alt="favicon"
-                                className="rounded-lg h-4 w-4"
-                              />
-                            )}
-                            <p className="text-xs text-black/50 dark:text-white/50 overflow-hidden whitespace-nowrap text-ellipsis">
-                              {source.metadata.url.replace(
-                                /.+\/\/|www.|\..+/g,
-                                '',
-                              )}
-                            </p>
-                          </div>
-                          <div className="flex flex-row items-center space-x-1 text-black/50 dark:text-white/50 text-xs">
-                            <div className="bg-black/50 dark:bg-white/50 h-[4px] w-[4px] rounded-full" />
-                            <span>{i + 1}</span>
-                          </div>
-                        </div>
-                      </a>
-                    ))}
-                  </div>
-                </DialogPanel>
-              </TransitionChild>
-            </div>
-          </div>
-        </Dialog>
-      </Transition>
    </div>
  );
 };
--- a/src/components/MessageTabs.tsx
+++ b/src/components/MessageTabs.tsx
@ -0,0 +1,535 @@
+/* eslint-disable @next/next/no-img-element */
+'use client';
+
+import { getSuggestions } from '@/lib/actions';
+import { cn } from '@/lib/utils';
+import {
+  BookCopy,
+  CheckCheck,
+  Copy as CopyIcon,
+  Disc3,
+  ImagesIcon,
+  Layers3,
+  Plus,
+  Sparkles,
+  StopCircle,
+  VideoIcon,
+  Volume2,
+} from 'lucide-react';
+import Markdown, { MarkdownToJSX } from 'markdown-to-jsx';
+import { useCallback, useEffect, useState } from 'react';
+import { Prism as SyntaxHighlighter } from 'react-syntax-highlighter';
+import { oneDark } from 'react-syntax-highlighter/dist/cjs/styles/prism';
+import { useSpeech } from 'react-text-to-speech';
+import { Message } from './ChatWindow';
+import Copy from './MessageActions/Copy';
+import ModelInfoButton from './MessageActions/ModelInfo';
+import Rewrite from './MessageActions/Rewrite';
+import MessageSources from './MessageSources';
+import SearchImages from './SearchImages';
+import SearchVideos from './SearchVideos';
+import ThinkBox from './ThinkBox';
+
+const ThinkTagProcessor = ({ children }: { children: React.ReactNode }) => {
+  return <ThinkBox content={children as string} />;
+};
+
+const CodeBlock = ({
+  className,
+  children,
+}: {
+  className?: string;
+  children: React.ReactNode;
+}) => {
+  // Extract language from className (format could be "language-javascript" or "lang-javascript")
+  let language = '';
+  if (className) {
+    if (className.startsWith('language-')) {
+      language = className.replace('language-', '');
+    } else if (className.startsWith('lang-')) {
+      language = className.replace('lang-', '');
+    }
+  }
+
+  const content = children as string;
+  const [isCopied, setIsCopied] = useState(false);
+
+  const handleCopyCode = () => {
+    navigator.clipboard.writeText(content);
+    setIsCopied(true);
+    setTimeout(() => setIsCopied(false), 2000);
+  };
+
+  return (
+    <div className="rounded-md overflow-hidden my-4 relative group border border-dark-secondary">
+      <div className="flex justify-between items-center px-4 py-2 bg-dark-200 border-b border-dark-secondary text-xs text-white/70 font-mono">
+        <span>{language}</span>
+        <button
+          onClick={handleCopyCode}
+          className="p-1 rounded-md hover:bg-dark-secondary transition duration-200"
+          aria-label="Copy code to clipboard"
+        >
+          {isCopied ? (
+            <CheckCheck size={14} className="text-green-500" />
+          ) : (
+            <CopyIcon size={14} className="text-white/70" />
+          )}
+        </button>
+      </div>
+      <SyntaxHighlighter
+        language={language || 'text'}
+        style={oneDark}
+        customStyle={{
+          margin: 0,
+          padding: '1rem',
+          borderRadius: 0,
+          backgroundColor: '#1c1c1c',
+        }}
+        wrapLines={true}
+        wrapLongLines={true}
+        showLineNumbers={language !== '' && content.split('\n').length > 1}
+        useInlineStyles={true}
+        PreTag="div"
+      >
+        {content}
+      </SyntaxHighlighter>
+    </div>
+  );
+};
+
+type TabType = 'text' | 'sources' | 'images' | 'videos';
+
+interface SearchTabsProps {
+  chatHistory: Message[];
+  query: string;
+  messageId: string;
+  message: Message;
+  isLast: boolean;
+  loading: boolean;
+  rewrite: (messageId: string) => void;
+  sendMessage: (
+    message: string,
+    options?: {
+      messageId?: string;
+      rewriteIndex?: number;
+      suggestions?: string[];
+    },
+  ) => void;
+}
+
+const MessageTabs = ({
+  chatHistory,
+  query,
+  messageId,
+  message,
+  isLast,
+  loading,
+  rewrite,
+  sendMessage,
+}: SearchTabsProps) => {
+  const [activeTab, setActiveTab] = useState<TabType>('text');
+  const [imageCount, setImageCount] = useState(0);
+  const [videoCount, setVideoCount] = useState(0);
+  const [parsedMessage, setParsedMessage] = useState(message.content);
+  const [speechMessage, setSpeechMessage] = useState(message.content);
+  const [loadingSuggestions, setLoadingSuggestions] = useState(false);
+  const { speechStatus, start, stop } = useSpeech({ text: speechMessage });
+
+  // Callback functions to update counts
+  const updateImageCount = (count: number) => {
+    setImageCount(count);
+  };
+
+  const updateVideoCount = (count: number) => {
+    setVideoCount(count);
+  };
+
+  // Load suggestions handling
+  const handleLoadSuggestions = useCallback(async () => {
+    if (
+      loadingSuggestions ||
+      (message?.suggestions && message.suggestions.length > 0)
+    )
+      return;
+
+    setLoadingSuggestions(true);
+    try {
+      const suggestions = await getSuggestions([...chatHistory, message]);
+      // Update the message.suggestions property through parent component
+      sendMessage('', { messageId: message.messageId, suggestions });
+    } catch (error) {
+      console.error('Error loading suggestions:', error);
+    } finally {
+      setLoadingSuggestions(false);
+    }
+  }, [loadingSuggestions, message, chatHistory, sendMessage]);
+
+  // Process message content
+  useEffect(() => {
+    const citationRegex = /\[([^\]]+)\]/g;
+    const regex = /\[(\d+)\]/g;
+    let processedMessage = message.content;
+
+    if (message.role === 'assistant' && message.content.includes('<think>')) {
+      const openThinkTag = processedMessage.match(/<think>/g)?.length || 0;
+      const closeThinkTag = processedMessage.match(/<\/think>/g)?.length || 0;
+
+      if (openThinkTag > closeThinkTag) {
+        processedMessage += '</think> <a> </a>'; // The extra <a> </a> is to prevent the think component from looking bad
+      }
+    }
+
+    if (
+      message.role === 'assistant' &&
+      message?.sources &&
+      message.sources.length > 0
+    ) {
+      setParsedMessage(
+        processedMessage.replace(
+          citationRegex,
+          (_, capturedContent: string) => {
+            const numbers = capturedContent
+              .split(',')
+              .map((numStr) => numStr.trim());
+
+            const linksHtml = numbers
+              .map((numStr) => {
+                const number = parseInt(numStr);
+
+                if (isNaN(number) || number <= 0) {
+                  return `[${numStr}]`;
+                }
+
+                const source = message.sources?.[number - 1];
+                const url = source?.metadata?.url;
+
+                if (url) {
+                  return `<a href="${url}" target="_blank" className="bg-light-secondary dark:bg-dark-secondary px-1 rounded ml-1 no-underline text-xs text-black/70 dark:text-white/70 relative">${numStr}</a>`;
+                } else {
+                  return `[${numStr}]`;
+                }
+              })
+              .join('');
+
+            return linksHtml;
+          },
+        ),
+      );
+      setSpeechMessage(message.content.replace(regex, ''));
+      return;
+    }
+
+    setSpeechMessage(message.content.replace(regex, ''));
+    setParsedMessage(processedMessage);
+  }, [message.content, message.sources, message.role]);
+
+  // Auto-suggest effect (similar to MessageBox)
+  useEffect(() => {
+    const autoSuggestions = localStorage.getItem('autoSuggestions');
+    if (
+      isLast &&
+      message.role === 'assistant' &&
+      !loading &&
+      autoSuggestions === 'true'
+    ) {
+      handleLoadSuggestions();
+    }
+  }, [isLast, loading, message.role, handleLoadSuggestions]);
+
+  // Markdown formatting options
+  const markdownOverrides: MarkdownToJSX.Options = {
+    overrides: {
+      think: {
+        component: ThinkTagProcessor,
+      },
+      code: {
+        component: ({ className, children }) => {
+          // Check if it's an inline code block or a fenced code block
+          if (className) {
+            // This is a fenced code block (```code```)
+            return <CodeBlock className={className}>{children}</CodeBlock>;
+          }
+          // This is an inline code block (`code`)
+          return (
+            <code className="px-1.5 py-0.5 rounded bg-dark-secondary text-white font-mono text-sm">
+              {children}
+            </code>
+          );
+        },
+      },
+      pre: {
+        component: ({ children }) => children,
+      },
+    },
+  };
+
+  return (
+    <div className="flex flex-col w-full">
+      {/* Tabs */}
+      <div className="flex border-b border-light-200 dark:border-dark-200 overflow-x-auto no-scrollbar sticky top-0 bg-light-primary dark:bg-dark-primary z-10 -mx-4 px-4 mb-2 shadow-sm">
+        <button
+          onClick={() => setActiveTab('text')}
+          className={cn(
+            'flex items-center px-4 py-3 text-sm font-medium transition-all duration-200 relative',
+            activeTab === 'text'
+              ? 'border-b-2 border-[#24A0ED] text-[#24A0ED] bg-light-100 dark:bg-dark-100'
+              : 'text-black/70 dark:text-white/70 hover:text-black dark:hover:text-white hover:bg-light-100 dark:hover:bg-dark-100',
+          )}
+          aria-selected={activeTab === 'text'}
+          role="tab"
+        >
+          <Disc3 size={16} className="mr-2" />
+          <span className="whitespace-nowrap">Answer</span>
+        </button>
+
+        {message.sources && message.sources.length > 0 && (
+          <button
+            onClick={() => setActiveTab('sources')}
+            className={cn(
+              'flex items-center space-x-2 px-4 py-3 text-sm font-medium transition-all duration-200 relative',
+              activeTab === 'sources'
+                ? 'border-b-2 border-[#24A0ED] text-[#24A0ED] bg-light-100 dark:bg-dark-100'
+                : 'text-black/70 dark:text-white/70 hover:text-black dark:hover:text-white hover:bg-light-100 dark:hover:bg-dark-100',
+            )}
+            aria-selected={activeTab === 'sources'}
+            role="tab"
+          >
+            <BookCopy size={16} />
+            <span className="whitespace-nowrap">Sources</span>
+            <span
+              className={cn(
+                'ml-1.5 px-1.5 py-0.5 text-xs rounded-full',
+                activeTab === 'sources'
+                  ? 'bg-[#24A0ED]/20 text-[#24A0ED]'
+                  : 'bg-light-200 dark:bg-dark-200 text-black/70 dark:text-white/70',
+              )}
+            >
+              {message.sources.length}
+            </span>
+          </button>
+        )}
+
+        <button
+          onClick={() => setActiveTab('images')}
+          className={cn(
+            'flex items-center space-x-2 px-4 py-3 text-sm font-medium transition-all duration-200 relative',
+            activeTab === 'images'
+              ? 'border-b-2 border-[#24A0ED] text-[#24A0ED] bg-light-100 dark:bg-dark-100'
+              : 'text-black/70 dark:text-white/70 hover:text-black dark:hover:text-white hover:bg-light-100 dark:hover:bg-dark-100',
+          )}
+          aria-selected={activeTab === 'images'}
+          role="tab"
+        >
+          <ImagesIcon size={16} />
+          <span className="whitespace-nowrap">Images</span>
+          {imageCount > 0 && (
+            <span
+              className={cn(
+                'ml-1.5 px-1.5 py-0.5 text-xs rounded-full',
+                activeTab === 'images'
+                  ? 'bg-[#24A0ED]/20 text-[#24A0ED]'
+                  : 'bg-light-200 dark:bg-dark-200 text-black/70 dark:text-white/70',
+              )}
+            >
+              {imageCount}
+            </span>
+          )}
+        </button>
+
+        <button
+          onClick={() => setActiveTab('videos')}
+          className={cn(
+            'flex items-center space-x-2 px-4 py-3 text-sm font-medium transition-all duration-200 relative',
+            activeTab === 'videos'
+              ? 'border-b-2 border-[#24A0ED] text-[#24A0ED] bg-light-100 dark:bg-dark-100'
+              : 'text-black/70 dark:text-white/70 hover:text-black dark:hover:text-white hover:bg-light-100 dark:hover:bg-dark-100',
+          )}
+          aria-selected={activeTab === 'videos'}
+          role="tab"
+        >
+          <VideoIcon size={16} />
+          <span className="whitespace-nowrap">Videos</span>
+          {videoCount > 0 && (
+            <span
+              className={cn(
+                'ml-1.5 px-1.5 py-0.5 text-xs rounded-full',
+                activeTab === 'videos'
+                  ? 'bg-[#24A0ED]/20 text-[#24A0ED]'
+                  : 'bg-light-200 dark:bg-dark-200 text-black/70 dark:text-white/70',
+              )}
+            >
+              {videoCount}
+            </span>
+          )}
+        </button>
+      </div>
+
+      {/* Tab content */}
+      <div
+        className="min-h-[150px] transition-all duration-200 ease-in-out"
+        role="tabpanel"
+      >
+        {/* Answer Tab */}
+        {activeTab === 'text' && (
+          <div className="flex flex-col space-y-4 animate-fadeIn">
+            <Markdown
+              className={cn(
+                'prose prose-h1:mb-3 prose-h2:mb-2 prose-h2:mt-6 prose-h2:font-[800] prose-h3:mt-4 prose-h3:mb-1.5 prose-h3:font-[600] prose-invert prose-p:leading-relaxed prose-pre:p-0 font-[400]',
+                'prose-code:bg-transparent prose-code:p-0 prose-code:text-inherit prose-code:font-normal prose-code:before:content-none prose-code:after:content-none',
+                'prose-pre:bg-transparent prose-pre:border-0 prose-pre:m-0 prose-pre:p-0',
+                'max-w-none break-words px-4 text-black dark:text-white',
+              )}
+              options={markdownOverrides}
+            >
+              {parsedMessage}
+            </Markdown>
+
+            {loading && isLast ? null : (
+              <div className="flex flex-row items-center justify-between w-full text-black dark:text-white px-4 py-4">
+                <div className="flex flex-row items-center space-x-1">
+                  <Rewrite rewrite={rewrite} messageId={message.messageId} />
+                  {message.modelStats && (
+                    <ModelInfoButton modelStats={message.modelStats} />
+                  )}
+                </div>
+                <div className="flex flex-row items-center space-x-1">
+                  <Copy initialMessage={message.content} message={message} />
+                  <button
+                    onClick={() => {
+                      if (speechStatus === 'started') {
+                        stop();
+                      } else {
+                        start();
+                      }
+                    }}
+                    className="p-2 text-black/70 dark:text-white/70 rounded-xl hover:bg-light-secondary dark:hover:bg-dark-secondary transition duration-200 hover:text-black dark:hover:text-white"
+                  >
+                    {speechStatus === 'started' ? (
+                      <StopCircle size={18} />
+                    ) : (
+                      <Volume2 size={18} />
+                    )}
+                  </button>
+                </div>
+              </div>
+            )}
+
+            {isLast && message.role === 'assistant' && !loading && (
+              <>
+                <div className="border-t border-light-secondary dark:border-dark-secondary px-4 pt-4 mt-4">
+                  <div className="flex flex-row items-center space-x-2 mb-3">
+                    <Layers3 size={20} />
+                    <h3 className="text-xl font-medium">Related</h3>
+
+                    {(!message.suggestions ||
+                      message.suggestions.length === 0) && (
+                      <button
+                        onClick={handleLoadSuggestions}
+                        disabled={loadingSuggestions}
+                        className="px-4 py-2 flex flex-row items-center justify-center space-x-2 rounded-lg bg-light-secondary dark:bg-dark-secondary hover:bg-light-200 dark:hover:bg-dark-200 transition duration-200 text-black/70 dark:text-white/70 hover:text-black dark:hover:text-white"
+                      >
+                        {loadingSuggestions ? (
+                          <div className="w-4 h-4 border-2 border-t-transparent border-gray-400 dark:border-gray-500 rounded-full animate-spin" />
+                        ) : (
+                          <Sparkles size={16} />
+                        )}
+                        <span>
+                          {loadingSuggestions
+                            ? 'Loading suggestions...'
+                            : 'Load suggestions'}
+                        </span>
+                      </button>
+                    )}
+                  </div>
+
+                  {message.suggestions && message.suggestions.length > 0 && (
+                    <div className="flex flex-col space-y-3 mt-2">
+                      {message.suggestions.map((suggestion, i) => (
+                        <div
+                          className="flex flex-col space-y-3 text-sm"
+                          key={i}
+                        >
+                          <div className="h-px w-full bg-light-secondary dark:bg-dark-secondary" />
+                          <div
+                            onClick={() => {
+                              sendMessage(suggestion);
+                            }}
+                            className="cursor-pointer flex flex-row justify-between font-medium space-x-2 items-center"
+                          >
+                            <p className="transition duration-200 hover:text-[#24A0ED]">
+                              {suggestion}
+                            </p>
+                            <Plus
+                              size={20}
+                              className="text-[#24A0ED] flex-shrink-0"
+                            />
+                          </div>
+                        </div>
+                      ))}
+                    </div>
+                  )}
+                </div>
+              </>
+            )}
+          </div>
+        )}
+
+        {/* Sources Tab */}
+        {activeTab === 'sources' &&
+          message.sources &&
+          message.sources.length > 0 && (
+            <div className="p-4 animate-fadeIn">
+              {message.searchQuery && (
+                <div className="mb-4 text-sm bg-light-secondary dark:bg-dark-secondary rounded-lg p-3">
+                  <span className="font-medium text-black/70 dark:text-white/70">
+                    Search query:
+                  </span>{' '}
+                  {message.searchUrl ? (
+                    <a
+                      href={message.searchUrl}
+                      target="_blank"
+                      rel="noopener noreferrer"
+                      className="dark:text-white text-black hover:underline"
+                    >
+                      {message.searchQuery}
+                    </a>
+                  ) : (
+                    <span className="text-black dark:text-white">
+                      {message.searchQuery}
+                    </span>
+                  )}
+                </div>
+              )}
+              <MessageSources sources={message.sources} />
+            </div>
+          )}
+
+        {/* Images Tab */}
+        {activeTab === 'images' && (
+          <div className="p-3 animate-fadeIn">
+            <SearchImages
+              query={query}
+              chatHistory={chatHistory}
+              messageId={messageId}
+              onImagesLoaded={updateImageCount}
+            />
+          </div>
+        )}
+
+        {/* Videos Tab */}
+        {activeTab === 'videos' && (
+          <div className="p-3 animate-fadeIn">
+            <SearchVideos
+              query={query}
+              chatHistory={chatHistory}
+              messageId={messageId}
+              onVideosLoaded={updateVideoCount}
+            />
+          </div>
+        )}
+      </div>
+    </div>
+  );
+};
+
+export default MessageTabs;
--- a/src/components/SearchImages.tsx
+++ b/src/components/SearchImages.tsx
@ -1,6 +1,5 @@
 /* eslint-disable @next/next/no-img-element */
-import { ImagesIcon, PlusIcon } from 'lucide-react';
-import { useState } from 'react';
+import { useEffect, useRef, useState } from 'react';
 import Lightbox from 'yet-another-react-lightbox';
 import 'yet-another-react-lightbox/styles.css';
 import { Message } from './ChatWindow';
@ -15,30 +14,64 @@ const SearchImages = ({
  query,
  chatHistory,
  messageId,
+  onImagesLoaded,
 }: {
  query: string;
  chatHistory: Message[];
  messageId: string;
+  onImagesLoaded?: (count: number) => void;
 }) => {
  const [images, setImages] = useState<Image[] | null>(null);
-  const [loading, setLoading] = useState(false);
+  const [loading, setLoading] = useState(true);
  const [open, setOpen] = useState(false);
  const [slides, setSlides] = useState<any[]>([]);
+  const [displayLimit, setDisplayLimit] = useState(10); // Initially show only 10 images
+  const loadedMessageIdsRef = useRef<Set<string>>(new Set());

-  return (
-    <>
-      {!loading && images === null && (
-        <button
-          id={`search-images-${messageId}`}
-          onClick={async () => {
+  // Function to show more images when the Show More button is clicked
+  const handleShowMore = () => {
+    // If we're already showing all images, don't do anything
+    if (images && displayLimit >= images.length) return;
+
+    // Otherwise, increase the display limit by 10, or show all images
+    setDisplayLimit((prev) =>
+      images ? Math.min(prev + 10, images.length) : prev,
+    );
+  };
+
+  useEffect(() => {
+    // Skip fetching if images are already loaded for this message
+    if (loadedMessageIdsRef.current.has(messageId)) {
+      return;
+    }
+
+    const fetchImages = async () => {
+      // Mark as loaded to prevent refetching
+      loadedMessageIdsRef.current.add(messageId);
      setLoading(true);

      const chatModelProvider = localStorage.getItem('chatModelProvider');
      const chatModel = localStorage.getItem('chatModel');
-
      const customOpenAIBaseURL = localStorage.getItem('openAIBaseURL');
      const customOpenAIKey = localStorage.getItem('openAIApiKey');
+      const ollamaContextWindow =
+        localStorage.getItem('ollamaContextWindow') || '2048';

+      // Get selected system prompt IDs from localStorage
+      const storedPromptIds = localStorage.getItem('selectedSystemPromptIds');
+      let selectedSystemPromptIds: string[] = [];
+      if (storedPromptIds) {
+        try {
+          selectedSystemPromptIds = JSON.parse(storedPromptIds);
+        } catch (e) {
+          console.error(
+            'Failed to parse selectedSystemPromptIds from localStorage',
+            e,
+          );
+        }
+      }
+
+      try {
        const res = await fetch(`/api/images`, {
          method: 'POST',
          headers: {
@ -54,7 +87,11 @@ const SearchImages = ({
                customOpenAIBaseURL: customOpenAIBaseURL,
                customOpenAIKey: customOpenAIKey,
              }),
+              ...(chatModelProvider === 'ollama' && {
+                ollamaContextWindow: parseInt(ollamaContextWindow),
+              }),
            },
+            selectedSystemPromptIds: selectedSystemPromptIds,
          }),
        });

@ -69,17 +106,21 @@ const SearchImages = ({
            };
          }),
        );
+        if (onImagesLoaded && images.length > 0) {
+          onImagesLoaded(images.length);
+        }
+      } catch (error) {
+        console.error('Error fetching images:', error);
+      } finally {
        setLoading(false);
-          }}
-          className="border border-dashed border-light-200 dark:border-dark-200 hover:bg-light-200 dark:hover:bg-dark-200 active:scale-95 duration-200 transition px-4 py-2 flex flex-row items-center justify-between rounded-lg dark:text-white text-sm w-full"
-        >
-          <div className="flex flex-row items-center space-x-2">
-            <ImagesIcon size={17} />
-            <p>Search images</p>
-          </div>
-          <PlusIcon className="text-[#24A0ED]" size={17} />
-        </button>
-      )}
+      }
+    };
+
+    fetchImages();
+  }, [query, messageId, chatHistory, onImagesLoaded]);
+
+  return (
+    <>
      {loading && (
        <div className="grid grid-cols-2 gap-2">
          {[...Array(4)].map((_, i) => (
@ -92,61 +133,40 @@ const SearchImages = ({
      )}
      {images !== null && images.length > 0 && (
        <>
-          <div className="grid grid-cols-2 gap-2">
-            {images.length > 4
-              ? images.slice(0, 3).map((image, i) => (
-                  <img
-                    onClick={() => {
-                      setOpen(true);
-                      setSlides([
-                        slides[i],
-                        ...slides.slice(0, i),
-                        ...slides.slice(i + 1),
-                      ]);
-                    }}
-                    key={i}
-                    src={image.img_src}
-                    alt={image.title}
-                    className="h-full w-full aspect-video object-cover rounded-lg transition duration-200 active:scale-95 hover:scale-[1.02] cursor-zoom-in"
-                  />
-                ))
-              : images.map((image, i) => (
-                  <img
-                    onClick={() => {
-                      setOpen(true);
-                      setSlides([
-                        slides[i],
-                        ...slides.slice(0, i),
-                        ...slides.slice(i + 1),
-                      ]);
-                    }}
-                    key={i}
-                    src={image.img_src}
-                    alt={image.title}
-                    className="h-full w-full aspect-video object-cover rounded-lg transition duration-200 active:scale-95 hover:scale-[1.02] cursor-zoom-in"
-                  />
-                ))}
-            {images.length > 4 && (
-              <button
-                onClick={() => setOpen(true)}
-                className="bg-light-100 hover:bg-light-200 dark:bg-dark-100 dark:hover:bg-dark-200 transition duration-200 active:scale-95 hover:scale-[1.02] h-auto w-full rounded-lg flex flex-col justify-between text-white p-2"
+          <div
+            className="grid grid-cols-2 gap-2"
+            key={`image-results-${messageId}`}
          >
-                <div className="flex flex-row items-center space-x-1">
-                  {images.slice(3, 6).map((image, i) => (
+            {images.slice(0, displayLimit).map((image, i) => (
              <img
+                onClick={() => {
+                  setOpen(true);
+                  setSlides([
+                    slides[i],
+                    ...slides.slice(0, i),
+                    ...slides.slice(i + 1),
+                  ]);
+                }}
                key={i}
                src={image.img_src}
                alt={image.title}
-                      className="h-6 w-12 rounded-md lg:h-3 lg:w-6 lg:rounded-sm aspect-video object-cover"
+                className="h-full w-full aspect-video object-cover rounded-lg transition duration-200 active:scale-95 hover:scale-[1.02] cursor-zoom-in"
              />
            ))}
          </div>
-                <p className="text-black/70 dark:text-white/70 text-xs">
-                  View {images.length - 3} more
-                </p>
+          {images.length > displayLimit && (
+            <div className="flex justify-center mt-4">
+              <button
+                onClick={handleShowMore}
+                className="px-4 py-2 bg-light-secondary dark:bg-dark-secondary hover:bg-light-200 dark:hover:bg-dark-200 text-black/70 dark:text-white/70 hover:text-black dark:hover:text-white rounded-md transition duration-200 flex items-center space-x-2"
+              >
+                <span>Show More Images</span>
+                <span className="text-sm opacity-75">
+                  ({displayLimit} of {images.length})
+                </span>
              </button>
-            )}
            </div>
+          )}
          <Lightbox open={open} close={() => setOpen(false)} slides={slides} />
        </>
      )}
--- a/src/components/SearchVideos.tsx
+++ b/src/components/SearchVideos.tsx
@ -1,6 +1,6 @@
 /* eslint-disable @next/next/no-img-element */
-import { PlayCircle, PlayIcon, PlusIcon, VideoIcon } from 'lucide-react';
-import { useRef, useState } from 'react';
+import { PlayCircle } from 'lucide-react';
+import { useEffect, useRef, useState } from 'react';
 import Lightbox, { GenericSlide, VideoSlide } from 'yet-another-react-lightbox';
 import 'yet-another-react-lightbox/styles.css';
 import { Message } from './ChatWindow';
@ -28,32 +28,64 @@ const Searchvideos = ({
  query,
  chatHistory,
  messageId,
+  onVideosLoaded,
 }: {
  query: string;
  chatHistory: Message[];
  messageId: string;
+  onVideosLoaded?: (count: number) => void;
 }) => {
  const [videos, setVideos] = useState<Video[] | null>(null);
-  const [loading, setLoading] = useState(false);
+  const [loading, setLoading] = useState(true);
  const [open, setOpen] = useState(false);
  const [slides, setSlides] = useState<VideoSlide[]>([]);
  const [currentIndex, setCurrentIndex] = useState(0);
+  const [displayLimit, setDisplayLimit] = useState(10); // Initially show only 10 videos
  const videoRefs = useRef<(HTMLIFrameElement | null)[]>([]);
+  const loadedMessageIdsRef = useRef<Set<string>>(new Set());

-  return (
-    <>
-      {!loading && videos === null && (
-        <button
-          id={`search-videos-${messageId}`}
-          onClick={async () => {
+  // Function to show more videos when the Show More button is clicked
+  const handleShowMore = () => {
+    // If we're already showing all videos, don't do anything
+    if (videos && displayLimit >= videos.length) return;
+
+    // Otherwise, increase the display limit by 10, or show all videos
+    setDisplayLimit((prev) =>
+      videos ? Math.min(prev + 10, videos.length) : prev,
+    );
+  };
+
+  useEffect(() => {
+    // Skip fetching if videos are already loaded for this message
+    if (loadedMessageIdsRef.current.has(messageId)) {
+      return;
+    }
+
+    const fetchVideos = async () => {
      setLoading(true);

      const chatModelProvider = localStorage.getItem('chatModelProvider');
      const chatModel = localStorage.getItem('chatModel');
-
      const customOpenAIBaseURL = localStorage.getItem('openAIBaseURL');
      const customOpenAIKey = localStorage.getItem('openAIApiKey');
+      const ollamaContextWindow =
+        localStorage.getItem('ollamaContextWindow') || '2048';

+      // Get selected system prompt IDs from localStorage
+      const storedPromptIds = localStorage.getItem('selectedSystemPromptIds');
+      let selectedSystemPromptIds: string[] = [];
+      if (storedPromptIds) {
+        try {
+          selectedSystemPromptIds = JSON.parse(storedPromptIds);
+        } catch (e) {
+          console.error(
+            'Failed to parse selectedSystemPromptIds from localStorage',
+            e,
+          );
+        }
+      }
+
+      try {
        const res = await fetch(`/api/videos`, {
          method: 'POST',
          headers: {
@ -69,7 +101,11 @@ const Searchvideos = ({
                customOpenAIBaseURL: customOpenAIBaseURL,
                customOpenAIKey: customOpenAIKey,
              }),
+              ...(chatModelProvider === 'ollama' && {
+                ollamaContextWindow: parseInt(ollamaContextWindow),
+              }),
            },
+            selectedSystemPromptIds: selectedSystemPromptIds,
          }),
        });

@ -86,17 +122,23 @@ const Searchvideos = ({
            };
          }),
        );
+        if (onVideosLoaded && videos.length > 0) {
+          onVideosLoaded(videos.length);
+        }
+        // Mark as loaded to prevent refetching
+        loadedMessageIdsRef.current.add(messageId);
+      } catch (error) {
+        console.error('Error fetching videos:', error);
+      } finally {
        setLoading(false);
-          }}
-          className="border border-dashed border-light-200 dark:border-dark-200 hover:bg-light-200 dark:hover:bg-dark-200 active:scale-95 duration-200 transition px-4 py-2 flex flex-row items-center justify-between rounded-lg dark:text-white text-sm w-full"
-        >
-          <div className="flex flex-row items-center space-x-2">
-            <VideoIcon size={17} />
-            <p>Search videos</p>
-          </div>
-          <PlusIcon className="text-[#24A0ED]" size={17} />
-        </button>
-      )}
+      }
+    };
+
+    fetchVideos();
+  }, [query, messageId, chatHistory, onVideosLoaded]);
+
+  return (
+    <>
      {loading && (
        <div className="grid grid-cols-2 gap-2">
          {[...Array(4)].map((_, i) => (
@ -109,33 +151,11 @@ const Searchvideos = ({
      )}
      {videos !== null && videos.length > 0 && (
        <>
-          <div className="grid grid-cols-2 gap-2">
-            {videos.length > 4
-              ? videos.slice(0, 3).map((video, i) => (
          <div
-                    onClick={() => {
-                      setOpen(true);
-                      setSlides([
-                        slides[i],
-                        ...slides.slice(0, i),
-                        ...slides.slice(i + 1),
-                      ]);
-                    }}
-                    className="relative transition duration-200 active:scale-95 hover:scale-[1.02] cursor-pointer"
-                    key={i}
+            className="grid grid-cols-2 gap-2"
+            key={`video-results-${messageId}`}
          >
-                    <img
-                      src={video.img_src}
-                      alt={video.title}
-                      className="relative h-full w-full aspect-video object-cover rounded-lg"
-                    />
-                    <div className="absolute bg-white/70 dark:bg-black/70 text-black/70 dark:text-white/70 px-2 py-1 flex flex-row items-center space-x-1 bottom-1 right-1 rounded-md">
-                      <PlayCircle size={15} />
-                      <p className="text-xs">Video</p>
-                    </div>
-                  </div>
-                ))
-              : videos.map((video, i) => (
+            {videos.slice(0, displayLimit).map((video, i) => (
              <div
                onClick={() => {
                  setOpen(true);
@ -159,27 +179,20 @@ const Searchvideos = ({
                </div>
              </div>
            ))}
-            {videos.length > 4 && (
+          </div>
+          {videos.length > displayLimit && (
+            <div className="flex justify-center mt-4">
              <button
-                onClick={() => setOpen(true)}
-                className="bg-light-100 hover:bg-light-200 dark:bg-dark-100 dark:hover:bg-dark-200 transition duration-200 active:scale-95 hover:scale-[1.02] h-auto w-full rounded-lg flex flex-col justify-between text-white p-2"
+                onClick={handleShowMore}
+                className="px-4 py-2 bg-light-secondary dark:bg-dark-secondary hover:bg-light-200 dark:hover:bg-dark-200 text-black/70 dark:text-white/70 hover:text-black dark:hover:text-white rounded-md transition duration-200 flex items-center space-x-2"
              >
-                <div className="flex flex-row items-center space-x-1">
-                  {videos.slice(3, 6).map((video, i) => (
-                    <img
-                      key={i}
-                      src={video.img_src}
-                      alt={video.title}
-                      className="h-6 w-12 rounded-md lg:h-3 lg:w-6 lg:rounded-sm aspect-video object-cover"
-                    />
-                  ))}
-                </div>
-                <p className="text-black/70 dark:text-white/70 text-xs">
-                  View {videos.length - 3} more
-                </p>
+                <span>Show More Videos</span>
+                <span className="text-sm opacity-75">
+                  ({displayLimit} of {videos.length})
+                </span>
              </button>
-            )}
            </div>
+          )}
          <Lightbox
            open={open}
            close={() => setOpen(false)}
--- a/src/instrumentation.ts
+++ b/src/instrumentation.ts
@ -0,0 +1,6 @@
+export async function register() {
+  if (process.env.NEXT_RUNTIME === 'nodejs') {
+    // Import error suppression when the server starts
+    await import('./lib/utils/errorSuppression');
+  }
+}
--- a/src/lib/actions.ts
+++ b/src/lib/actions.ts
@ -6,6 +6,22 @@ export const getSuggestions = async (chatHisory: Message[]) => {

  const customOpenAIKey = localStorage.getItem('openAIApiKey');
  const customOpenAIBaseURL = localStorage.getItem('openAIBaseURL');
+  const ollamaContextWindow =
+    localStorage.getItem('ollamaContextWindow') || '2048';
+
+  // Get selected system prompt IDs from localStorage
+  const storedPromptIds = localStorage.getItem('selectedSystemPromptIds');
+  let selectedSystemPromptIds: string[] = [];
+  if (storedPromptIds) {
+    try {
+      selectedSystemPromptIds = JSON.parse(storedPromptIds);
+    } catch (e) {
+      console.error(
+        'Failed to parse selectedSystemPromptIds from localStorage',
+        e,
+      );
+    }
+  }

  const res = await fetch(`/api/suggestions`, {
    method: 'POST',
@ -21,7 +37,11 @@ export const getSuggestions = async (chatHisory: Message[]) => {
          customOpenAIKey,
          customOpenAIBaseURL,
        }),
+        ...(chatModelProvider === 'ollama' && {
+          ollamaContextWindow: parseInt(ollamaContextWindow),
+        }),
      },
+      selectedSystemPromptIds: selectedSystemPromptIds,
    }),
  });

--- a/src/lib/agents/agentState.ts
+++ b/src/lib/agents/agentState.ts
@ -0,0 +1,81 @@
+import { BaseMessage } from '@langchain/core/messages';
+import { Annotation, END } from '@langchain/langgraph';
+import { Document } from 'langchain/document';
+
+/**
+ * State interface for the agent supervisor workflow
+ */
+export const AgentState = Annotation.Root({
+  messages: Annotation<BaseMessage[]>({
+    reducer: (x, y) => x.concat(y),
+    default: () => [],
+  }),
+  query: Annotation<string>({
+    reducer: (x, y) => y ?? x,
+    default: () => '',
+  }),
+  relevantDocuments: Annotation<Document[]>({
+    reducer: (x, y) => x.concat(y),
+    default: () => [],
+  }),
+  bannedSummaryUrls: Annotation<string[]>({
+    reducer: (x, y) => x.concat(y),
+    default: () => [],
+  }),
+  bannedPreviewUrls: Annotation<string[]>({
+    reducer: (x, y) => x.concat(y),
+    default: () => [],
+  }),
+  searchInstructionHistory: Annotation<string[]>({
+    reducer: (x, y) => x.concat(y),
+    default: () => [],
+  }),
+  searchInstructions: Annotation<string>({
+    reducer: (x, y) => y ?? x,
+    default: () => '',
+  }),
+  next: Annotation<string>({
+    reducer: (x, y) => y ?? x ?? END,
+    default: () => END,
+  }),
+  analysis: Annotation<string>({
+    reducer: (x, y) => y ?? x,
+    default: () => '',
+  }),
+  fullAnalysisAttempts: Annotation<number>({
+    reducer: (x, y) => (y ?? 0) + x,
+    default: () => 0,
+  }),
+  tasks: Annotation<string[]>({
+    reducer: (x, y) => y ?? x,
+    default: () => [],
+  }),
+  currentTaskIndex: Annotation<number>({
+    reducer: (x, y) => y ?? x,
+    default: () => 0,
+  }),
+  originalQuery: Annotation<string>({
+    reducer: (x, y) => y ?? x,
+    default: () => '',
+  }),
+  fileIds: Annotation<string[]>({
+    reducer: (x, y) => y ?? x,
+    default: () => [],
+  }),
+  focusMode: Annotation<string>({
+    reducer: (x, y) => y ?? x,
+    default: () => 'webSearch',
+  }),
+  urlsToSummarize: Annotation<string[]>({
+    reducer: (x, y) => y ?? x,
+    default: () => [],
+  }),
+  summarizationIntent: Annotation<string>({
+    reducer: (x, y) => y ?? x,
+    default: () => '',
+  }),
+  recursionLimitReached: Annotation<boolean>({
+    reducer: (x, y) => y ?? x,
+    default: () => false,
+  }),
+});
--- a/src/lib/agents/analyzerAgent.ts
+++ b/src/lib/agents/analyzerAgent.ts
@ -0,0 +1,360 @@
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import {
+  AIMessage,
+  HumanMessage,
+  SystemMessage,
+} from '@langchain/core/messages';
+import { ChatPromptTemplate } from '@langchain/core/prompts';
+import { Command, END } from '@langchain/langgraph';
+import { EventEmitter } from 'events';
+import { z } from 'zod';
+import LineOutputParser from '../outputParsers/lineOutputParser';
+import { formatDateForLLM } from '../utils';
+import { AgentState } from './agentState';
+import { setTemperature } from '../utils/modelUtils';
+import {
+  additionalUserInputPrompt,
+  additionalWebSearchPrompt,
+  decideNextActionPrompt,
+} from '../prompts/analyzer';
+import {
+  removeThinkingBlocks,
+  removeThinkingBlocksFromMessages,
+} from '../utils/contentUtils';
+import { withStructuredOutput } from '../utils/structuredOutput';
+import next from 'next';
+
+// Define Zod schemas for structured output
+const NextActionSchema = z.object({
+  action: z
+    .enum(['good_content', 'need_user_info', 'need_more_info'])
+    .describe('The next action to take based on content analysis'),
+  reasoning: z
+    .string()
+    .describe('Brief explanation of why this action was chosen'),
+});
+
+const UserInfoRequestSchema = z.object({
+  question: z
+    .string()
+    .describe('A detailed question to ask the user for additional information'),
+  reasoning: z
+    .string()
+    .describe('Explanation of why this information is needed'),
+});
+
+const SearchRefinementSchema = z.object({
+  question: z
+    .string()
+    .describe('A refined search question to gather more specific information'),
+  reasoning: z
+    .string()
+    .describe(
+      'Explanation of what information is missing and why this search will help',
+    ),
+});
+
+export class AnalyzerAgent {
+  private llm: BaseChatModel;
+  private emitter: EventEmitter;
+  private systemInstructions: string;
+  private signal: AbortSignal;
+
+  constructor(
+    llm: BaseChatModel,
+    emitter: EventEmitter,
+    systemInstructions: string,
+    signal: AbortSignal,
+  ) {
+    this.llm = llm;
+    this.emitter = emitter;
+    this.systemInstructions = systemInstructions;
+    this.signal = signal;
+  }
+
+  async execute(state: typeof AgentState.State): Promise<Command> {
+    try {
+      //setTemperature(this.llm, 0.0);
+
+      // Initialize originalQuery if not set
+      if (!state.originalQuery) {
+        state.originalQuery = state.query;
+      }
+
+      // Check for URLs first - if found and not yet processed, route to URL summarization
+      if (!state.urlsToSummarize || state.urlsToSummarize.length === 0) {
+        const urlRegex = /https?:\/\/[^\s]+/gi;
+        const urls = [...new Set(state.query.match(urlRegex) || [])];
+
+        if (urls.length > 0) {
+          console.log(
+            'URLs detected in initial query, routing to URL summarization',
+          );
+          console.log(`URLs found: ${urls.join(', ')}`);
+
+          // Emit URL detection event
+          this.emitter.emit('agent_action', {
+            type: 'agent_action',
+            data: {
+              action: 'URLS_DETECTED_ROUTING',
+              message: `Detected ${urls.length} URL(s) in query - processing content first`,
+              details: {
+                query: state.query,
+                urls: urls,
+              },
+            },
+          });
+
+          return new Command({
+            goto: 'url_summarization',
+            update: {
+              urlsToSummarize: urls,
+              summarizationIntent: `Process the content from the provided URLs to help answer: ${state.query}`,
+            },
+          });
+        }
+      }
+
+      // Skip full analysis if this is the first run.
+      //if (state.fullAnalysisAttempts > 0) {
+      // Emit initial analysis event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'ANALYZING_CONTEXT',
+          message:
+            'Analyzing the context to see if we have enough information to answer the query',
+          details: {
+            documentCount: state.relevantDocuments.length,
+            query: state.query,
+            searchIterations: state.searchInstructionHistory.length,
+          },
+        },
+      });
+
+      console.log(
+        `Analyzing ${state.relevantDocuments.length} documents for relevance...`,
+      );
+
+      const nextActionPrompt = await ChatPromptTemplate.fromTemplate(
+        decideNextActionPrompt,
+      ).format({
+        systemInstructions: this.systemInstructions,
+        context: state.relevantDocuments
+          .map(
+            (doc, index) =>
+              `<source${index + 1}>${doc?.metadata?.title ? `<title>${doc?.metadata?.title}</title>` : ''}${doc?.metadata.url ? `<url>${doc?.metadata?.url}</url>` : ''}<content>${doc.pageContent}</content></source${index + 1}>`,
+          )
+          .join('\n\n'),
+        date: formatDateForLLM(new Date()),
+        searchInstructionHistory: state.searchInstructionHistory
+          .map((question) => `- ${question}`)
+          .join('\n'),
+        query: state.originalQuery || state.query, // Use original query for analysis context
+      });
+
+      const thinkingBlocksRemovedMessages = removeThinkingBlocksFromMessages(
+        state.messages,
+      );
+
+      // Use structured output for next action decision
+      const structuredLlm = withStructuredOutput(this.llm, NextActionSchema, {
+        name: 'analyze_content',
+      });
+
+      const nextActionResponse = await structuredLlm.invoke(
+        [...thinkingBlocksRemovedMessages, new HumanMessage(nextActionPrompt)],
+        { signal: this.signal },
+      );
+
+      console.log('Next action response:', nextActionResponse);
+
+      if (nextActionResponse.action !== 'good_content') {
+        // If we don't have enough information, but we still have available tasks, proceed with the next task
+
+        if (state.tasks && state.tasks.length > 0) {
+          const hasMoreTasks = state.currentTaskIndex < state.tasks.length - 1;
+
+          if (hasMoreTasks) {
+            return new Command({
+              goto: 'task_manager',
+            });
+          }
+        }
+
+        if (nextActionResponse.action === 'need_user_info') {
+          // Use structured output for user info request
+          const userInfoLlm = withStructuredOutput(
+            this.llm,
+            UserInfoRequestSchema,
+            {
+              name: 'request_user_info',
+            },
+          );
+
+          const moreUserInfoPrompt = await ChatPromptTemplate.fromTemplate(
+            additionalUserInputPrompt,
+          ).format({
+            systemInstructions: this.systemInstructions,
+            context: state.relevantDocuments
+              .map(
+                (doc, index) =>
+                  `<source${index + 1}>${doc?.metadata?.title ? `<title>${doc?.metadata?.title}</title>` : ''}<content>${doc.pageContent}</content></source${index + 1}>`,
+              )
+              .join('\n\n'),
+            date: formatDateForLLM(new Date()),
+            searchInstructionHistory: state.searchInstructionHistory
+              .map((question) => `- ${question}`)
+              .join('\n'),
+            query: state.originalQuery || state.query, // Use original query for user info context
+            previousAnalysis: nextActionResponse.reasoning, // Include reasoning from previous analysis
+          });
+
+          const userInfoRequest = await userInfoLlm.invoke(
+            [
+              ...removeThinkingBlocksFromMessages(state.messages),
+              new HumanMessage(moreUserInfoPrompt),
+            ],
+            { signal: this.signal },
+          );
+
+          // Emit the complete question to the user
+          this.emitter.emit(
+            'data',
+            JSON.stringify({
+              type: 'response',
+              data: userInfoRequest.question,
+            }),
+          );
+
+          this.emitter.emit('end');
+
+          // Create the final response message with the complete content
+          const response = new SystemMessage(userInfoRequest.question);
+
+          return new Command({
+            goto: END,
+            update: {
+              messages: [response],
+            },
+          });
+        }
+
+        // If we need more information from the LLM, generate a more specific search query
+        // Use structured output for search refinement
+        const searchRefinementLlm = withStructuredOutput(
+          this.llm,
+          SearchRefinementSchema,
+          {
+            name: 'refine_search',
+          },
+        );
+
+        const moreInfoPrompt = await ChatPromptTemplate.fromTemplate(
+          additionalWebSearchPrompt,
+        ).format({
+          systemInstructions: this.systemInstructions,
+          context: state.relevantDocuments
+            .map(
+              (doc, index) =>
+                `<source${index + 1}>${doc?.metadata?.title ? `\n<title>${doc?.metadata?.title}</title>` : ''}\n<content>${doc.pageContent}</content>\n</source${index + 1}>`,
+            )
+            .join('\n\n'),
+          date: formatDateForLLM(new Date()),
+          searchInstructionHistory: state.searchInstructionHistory
+            .map((question) => `- ${question}`)
+            .join('\n'),
+          query: state.originalQuery || state.query, // Use original query for more info context
+          previousAnalysis: nextActionResponse.reasoning, // Include reasoning from previous analysis
+        });
+
+        const searchRefinement = await searchRefinementLlm.invoke(
+          [
+            ...removeThinkingBlocksFromMessages(state.messages),
+            new HumanMessage(moreInfoPrompt),
+          ],
+          { signal: this.signal },
+        );
+
+        // Emit reanalyzing event when we need more information
+        this.emitter.emit('agent_action', {
+          type: 'agent_action',
+          data: {
+            action: 'MORE_DATA_NEEDED',
+            message:
+              'Current context is insufficient - analyzing search requirements',
+            details: {
+              nextSearchQuery: searchRefinement.question,
+              documentCount: state.relevantDocuments.length,
+              searchIterations: state.searchInstructionHistory.length,
+              query: state.originalQuery || state.query, // Show original query in details
+              currentSearchFocus: searchRefinement.question,
+            },
+          },
+        });
+
+        return new Command({
+          goto: 'task_manager',
+          update: {
+            // messages: [
+            //   new AIMessage(
+            //     `The following question can help refine the search: ${searchRefinement.question}`,
+            //   ),
+            // ],
+            query: searchRefinement.question, // Use the refined question for TaskManager to analyze
+            searchInstructions: searchRefinement.question,
+            searchInstructionHistory: [
+              ...(state.searchInstructionHistory || []),
+              searchRefinement.question,
+            ],
+            fullAnalysisAttempts: 1,
+            originalQuery: state.originalQuery || state.query, // Preserve the original user query
+            // Reset task list so TaskManager can break down the search requirements again
+            tasks: [],
+            currentTaskIndex: 0,
+          },
+        });
+      }
+
+      // Emit information gathering complete event when we have sufficient information
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'INFORMATION_GATHERING_COMPLETE',
+          message: 'Ready to respond.',
+          details: {
+            documentCount: state.relevantDocuments.length,
+            searchIterations: state.searchInstructionHistory.length,
+            totalTasks: state.tasks?.length || 1,
+            query: state.originalQuery || state.query,
+          },
+        },
+      });
+
+      return new Command({
+        goto: 'synthesizer',
+        // update: {
+        //   messages: [
+        //     new AIMessage(
+        //       `Analysis completed. We have sufficient information to answer the query.`,
+        //     ),
+        //   ],
+        // },
+      });
+    } catch (error) {
+      console.error('Analysis error:', error);
+      const errorMessage = new AIMessage(
+        `Analysis failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+
+      return new Command({
+        goto: END,
+        update: {
+          messages: [errorMessage],
+        },
+      });
+    } finally {
+      setTemperature(this.llm); // Reset temperature for subsequent actions
+    }
+  }
+}
--- a/src/lib/agents/contentRouterAgent.ts
+++ b/src/lib/agents/contentRouterAgent.ts
@ -0,0 +1,233 @@
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import { AIMessage } from '@langchain/core/messages';
+import { PromptTemplate } from '@langchain/core/prompts';
+import { Command, END } from '@langchain/langgraph';
+import { EventEmitter } from 'events';
+import { z } from 'zod';
+import fs from 'node:fs';
+import path from 'node:path';
+import { AgentState } from './agentState';
+import { contentRouterPrompt } from '../prompts/contentRouter';
+import { removeThinkingBlocksFromMessages } from '../utils/contentUtils';
+import { withStructuredOutput } from '../utils/structuredOutput';
+
+// Define Zod schema for structured router decision output
+const RouterDecisionSchema = z.object({
+  decision: z
+    .enum(['file_search', 'web_search', 'analyzer'])
+    .describe('The next step to take in the workflow'),
+  reasoning: z.string().describe('Explanation of why this decision was made'),
+});
+
+type RouterDecision = z.infer<typeof RouterDecisionSchema>;
+
+export class ContentRouterAgent {
+  private llm: BaseChatModel;
+  private emitter: EventEmitter;
+  private systemInstructions: string;
+  private signal: AbortSignal;
+
+  constructor(
+    llm: BaseChatModel,
+    emitter: EventEmitter,
+    systemInstructions: string,
+    signal: AbortSignal,
+  ) {
+    this.llm = llm;
+    this.emitter = emitter;
+    this.systemInstructions = systemInstructions;
+    this.signal = signal;
+  }
+
+  /**
+   * Content router agent node
+   */
+  async execute(state: typeof AgentState.State): Promise<Command> {
+    try {
+      // Determine current task to process
+      const currentTask =
+        state.tasks && state.tasks.length > 0
+          ? state.tasks[state.currentTaskIndex || 0]
+          : state.query;
+
+      console.log(
+        `Content router processing task ${(state.currentTaskIndex || 0) + 1} of ${state.tasks?.length || 1}: "${currentTask}"`,
+      );
+
+      // Extract focus mode from state - this should now come from the API
+      const focusMode = state.focusMode || 'webSearch';
+
+      const hasFiles = state.fileIds && state.fileIds.length > 0;
+      const documentCount = state.relevantDocuments.length;
+      const searchHistory = state.searchInstructionHistory.join(', ') || 'None';
+
+      // Extract file topics if files are available
+      const fileTopics = hasFiles
+        ? await this.extractFileTopics(state.fileIds!)
+        : 'None';
+
+      // Emit routing decision event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'ROUTING_DECISION',
+          message: `Determining optimal information source for current task`,
+          details: {
+            query: state.query,
+            currentTask: currentTask,
+            taskIndex: (state.currentTaskIndex || 0) + 1,
+            totalTasks: state.tasks?.length || 1,
+            focusMode: focusMode,
+            hasFiles: hasFiles,
+            fileCount: state.fileIds?.length || 0,
+            documentCount: documentCount,
+            searchIterations: state.searchInstructionHistory.length,
+          },
+        },
+      });
+
+      const template = PromptTemplate.fromTemplate(contentRouterPrompt);
+      const prompt = await template.format({
+        systemInstructions: this.systemInstructions,
+        currentTask: currentTask,
+        query: state.originalQuery || state.query,
+        focusMode: focusMode,
+        hasFiles: hasFiles,
+        fileTopics: fileTopics,
+        documentCount: documentCount,
+        searchHistory: searchHistory,
+      });
+
+      // Use structured output for routing decision
+      const structuredLlm = withStructuredOutput(
+        this.llm,
+        RouterDecisionSchema,
+        {
+          name: 'route_content',
+        },
+      );
+
+      const routerDecision = (await structuredLlm.invoke(
+        [...removeThinkingBlocksFromMessages(state.messages), prompt],
+        { signal: this.signal },
+      )) as RouterDecision;
+
+      console.log(`Router decision: ${routerDecision.decision}`);
+      console.log(`Router reasoning: ${routerDecision.reasoning}`);
+      console.log(`File topics: ${fileTopics}`);
+      console.log(`Focus mode: ${focusMode}`);
+
+      // Validate decision based on focus mode restrictions
+      const validatedDecision = this.validateDecision(
+        routerDecision,
+        focusMode,
+        hasFiles,
+      );
+
+      // Emit routing result event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'ROUTING_RESULT',
+          message: `Routing to ${validatedDecision.decision}: ${validatedDecision.reasoning}`,
+          details: {
+            query: state.query,
+            currentTask: currentTask,
+            taskIndex: (state.currentTaskIndex || 0) + 1,
+            totalTasks: state.tasks?.length || 1,
+            decision: validatedDecision.decision,
+            focusMode: focusMode,
+            hasFiles: hasFiles,
+            documentCount: documentCount,
+            searchIterations: state.searchInstructionHistory.length,
+          },
+        },
+      });
+
+      const responseMessage = `Content routing completed. Next step: ${validatedDecision.decision}`;
+      console.log(responseMessage);
+
+      return new Command({
+        goto: validatedDecision.decision,
+        // update: {
+        //   messages: [new AIMessage(responseMessage)],
+        // },
+      });
+    } catch (error) {
+      console.error('Content router error:', error);
+      const errorMessage = new AIMessage(
+        `Content routing failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+
+      return new Command({
+        goto: END,
+        update: {
+          messages: [errorMessage],
+        },
+      });
+    }
+  }
+
+  /**
+   * Extract semantic topics from attached files for relevance assessment
+   */
+  private async extractFileTopics(fileIds: string[]): Promise<string> {
+    try {
+      const topics = fileIds.map((fileId) => {
+        try {
+          const filePath = path.join(process.cwd(), 'uploads', fileId);
+          const contentPath = filePath + '-extracted.json';
+
+          if (fs.existsSync(contentPath)) {
+            const content = JSON.parse(fs.readFileSync(contentPath, 'utf8'));
+            const filename = content.title || 'Document';
+
+            // Use LLM-generated semantic topics if available, otherwise fall back to filename
+            const semanticTopics = content.topics;
+            return semanticTopics || filename;
+          }
+          return 'Unknown Document';
+        } catch (error) {
+          console.warn(`Error extracting topic for file ${fileId}:`, error);
+          return 'Unknown Document';
+        }
+      });
+
+      return topics.join('; ');
+    } catch (error) {
+      console.warn('Error extracting file topics:', error);
+      return 'Unable to determine file topics';
+    }
+  }
+
+  /**
+   * Validate and potentially override the router decision based on focus mode restrictions
+   */
+  private validateDecision(
+    decision: RouterDecision,
+    focusMode: string,
+    hasFiles: boolean,
+  ): RouterDecision {
+    // Enforce focus mode restrictions for chat and localResearch modes
+    if (
+      (focusMode === 'chat' || focusMode === 'localResearch') &&
+      decision.decision === 'web_search'
+    ) {
+      // Override to file_search if files are available, otherwise analyzer
+      const fallbackDecision = hasFiles ? 'file_search' : 'analyzer';
+
+      console.log(
+        `Overriding web_search decision to ${fallbackDecision} due to focus mode restriction: ${focusMode}`,
+      );
+
+      return {
+        decision: fallbackDecision as 'file_search' | 'analyzer',
+        reasoning: `Overridden to ${fallbackDecision} - web search not allowed in ${focusMode} mode. ${decision.reasoning}`,
+      };
+    }
+
+    // For webSearch mode, trust the LLM's decision about file relevance
+    // No overrides needed - the enhanced prompt handles file relevance assessment
+    return decision;
+  }
+}
--- a/src/lib/agents/fileSearchAgent.ts
+++ b/src/lib/agents/fileSearchAgent.ts
@ -0,0 +1,238 @@
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import { AIMessage } from '@langchain/core/messages';
+import { Command, END } from '@langchain/langgraph';
+import { EventEmitter } from 'events';
+import { Document } from 'langchain/document';
+import { AgentState } from './agentState';
+import { Embeddings } from '@langchain/core/embeddings';
+import {
+  processFilesToDocuments,
+  getRankedDocs,
+} from '../utils/fileProcessing';
+
+export class FileSearchAgent {
+  private llm: BaseChatModel;
+  private emitter: EventEmitter;
+  private systemInstructions: string;
+  private signal: AbortSignal;
+  private embeddings: Embeddings;
+
+  constructor(
+    llm: BaseChatModel,
+    emitter: EventEmitter,
+    systemInstructions: string,
+    signal: AbortSignal,
+    embeddings: Embeddings,
+  ) {
+    this.llm = llm;
+    this.emitter = emitter;
+    this.systemInstructions = systemInstructions;
+    this.signal = signal;
+    this.embeddings = embeddings;
+  }
+
+  /**
+   * File search agent node
+   */
+  async execute(state: typeof AgentState.State): Promise<Command> {
+    try {
+      // Determine current task to process
+      const currentTask =
+        state.tasks && state.tasks.length > 0
+          ? state.tasks[state.currentTaskIndex || 0]
+          : state.query;
+
+      console.log(
+        `Processing file search for task ${(state.currentTaskIndex || 0) + 1} of ${state.tasks?.length || 1}: "${currentTask}"`,
+      );
+
+      // Check if we have file IDs to process
+      if (!state.fileIds || state.fileIds.length === 0) {
+        console.log('No files attached for search');
+        return new Command({
+          goto: 'analyzer',
+          update: {
+            messages: [new AIMessage('No files attached to search.')],
+          },
+        });
+      }
+
+      // Emit consulting attached files event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'CONSULTING_ATTACHED_FILES',
+          message: `Consulting attached files...`,
+          details: {
+            query: state.query,
+            currentTask: currentTask,
+            taskIndex: (state.currentTaskIndex || 0) + 1,
+            totalTasks: state.tasks?.length || 1,
+            fileCount: state.fileIds.length,
+            documentCount: state.relevantDocuments.length,
+          },
+        },
+      });
+
+      // Process files to documents
+      const fileDocuments = await processFilesToDocuments(state.fileIds);
+
+      if (fileDocuments.length === 0) {
+        console.log('No processable file content found');
+        return new Command({
+          goto: 'analyzer',
+          // update: {
+          //   messages: [
+          //     new AIMessage('No searchable content found in attached files.'),
+          //   ],
+          // },
+        });
+      }
+
+      console.log(
+        `Processed ${fileDocuments.length} file documents for search`,
+      );
+
+      // Emit searching file content event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'SEARCHING_FILE_CONTENT',
+          message: `Searching through ${fileDocuments.length} file sections for relevant information`,
+          details: {
+            query: state.query,
+            currentTask: currentTask,
+            taskIndex: (state.currentTaskIndex || 0) + 1,
+            totalTasks: state.tasks?.length || 1,
+            fileDocumentCount: fileDocuments.length,
+            documentCount: state.relevantDocuments.length,
+          },
+        },
+      });
+
+      // Generate query embedding for similarity search
+      const queryEmbedding = await this.embeddings.embedQuery(
+        state.originalQuery + ' ' + currentTask,
+      );
+
+      // Perform similarity search over file documents
+      const rankedDocuments = getRankedDocs(
+        queryEmbedding,
+        fileDocuments,
+        12, // maxDocs
+        0.3, // similarity threshold
+      );
+
+      console.log(`Found ${rankedDocuments.length} relevant file sections`);
+
+      if (rankedDocuments.length === 0) {
+        // Emit no relevant content event
+        this.emitter.emit('agent_action', {
+          type: 'agent_action',
+          data: {
+            action: 'NO_RELEVANT_FILE_CONTENT',
+            message: `No relevant content found in attached files for the current task`,
+            details: {
+              query: state.query,
+              currentTask: currentTask,
+              taskIndex: (state.currentTaskIndex || 0) + 1,
+              totalTasks: state.tasks?.length || 1,
+              searchedDocuments: fileDocuments.length,
+              documentCount: state.relevantDocuments.length,
+            },
+          },
+        });
+
+        return new Command({
+          goto: 'analyzer',
+          // update: {
+          //   messages: [
+          //     new AIMessage(
+          //       'No relevant content found in attached files for the current task.',
+          //     ),
+          //   ],
+          // },
+        });
+      }
+
+      // Emit file content found event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'FILE_CONTENT_FOUND',
+          message: `Found ${rankedDocuments.length} relevant sections in attached files`,
+          details: {
+            query: state.query,
+            currentTask: currentTask,
+            taskIndex: (state.currentTaskIndex || 0) + 1,
+            totalTasks: state.tasks?.length || 1,
+            relevantSections: rankedDocuments.length,
+            searchedDocuments: fileDocuments.length,
+            documentCount:
+              state.relevantDocuments.length + rankedDocuments.length,
+          },
+        },
+      });
+
+      const responseMessage = `File search completed. Found ${rankedDocuments.length} relevant sections in attached files.`;
+      console.log(responseMessage);
+
+      return new Command({
+        goto: 'analyzer', // Route back to analyzer to process the results
+        update: {
+          // messages: [new AIMessage(responseMessage)],
+          relevantDocuments: rankedDocuments,
+        },
+      });
+    } catch (error) {
+      console.error('File search error:', error);
+      const errorMessage = new AIMessage(
+        `File search failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+
+      return new Command({
+        goto: END,
+        update: {
+          messages: [errorMessage],
+        },
+      });
+    }
+  }
+
+  /**
+   * Perform a similarity search over file documents
+   * @param state The current agent state
+   * @returns Ranked documents relevant to the current task
+   */
+  async search(state: typeof AgentState.State): Promise<Document[]> {
+    if (!state.fileIds || state.fileIds.length === 0) {
+      return [];
+    }
+
+    // Process files to documents
+    const fileDocuments = await processFilesToDocuments(state.fileIds);
+
+    if (fileDocuments.length === 0) {
+      return [];
+    }
+
+    // Determine current task to search for
+    const currentTask =
+      state.tasks && state.tasks.length > 0
+        ? state.tasks[state.currentTaskIndex || 0]
+        : state.query;
+
+    // Generate query embedding for similarity search
+    const queryEmbedding = await this.embeddings.embedQuery(
+      state.originalQuery + ' ' + currentTask,
+    );
+
+    // Perform similarity search and return ranked documents
+    return getRankedDocs(
+      queryEmbedding,
+      fileDocuments,
+      8, // maxDocs
+      0.3, // similarity threshold
+    );
+  }
+}
--- a/src/lib/agents/index.ts
+++ b/src/lib/agents/index.ts
@ -0,0 +1,8 @@
+export { AgentState } from './agentState';
+export { WebSearchAgent } from './webSearchAgent';
+export { AnalyzerAgent } from './analyzerAgent';
+export { SynthesizerAgent } from './synthesizerAgent';
+export { TaskManagerAgent } from './taskManagerAgent';
+export { FileSearchAgent } from './fileSearchAgent';
+export { ContentRouterAgent } from './contentRouterAgent';
+export { URLSummarizationAgent } from './urlSummarizationAgent';
--- a/src/lib/agents/synthesizerAgent.ts
+++ b/src/lib/agents/synthesizerAgent.ts
@ -0,0 +1,165 @@
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import { HumanMessage, SystemMessage } from '@langchain/core/messages';
+import { PromptTemplate } from '@langchain/core/prompts';
+import { Command, END } from '@langchain/langgraph';
+import { EventEmitter } from 'events';
+import { getModelName } from '../utils/modelUtils';
+import { AgentState } from './agentState';
+import { removeThinkingBlocksFromMessages } from '../utils/contentUtils';
+import { synthesizerPrompt } from '../prompts/synthesizer';
+
+export class SynthesizerAgent {
+  private llm: BaseChatModel;
+  private emitter: EventEmitter;
+  private personaInstructions: string;
+  private signal: AbortSignal;
+
+  constructor(
+    llm: BaseChatModel,
+    emitter: EventEmitter,
+    personaInstructions: string,
+    signal: AbortSignal,
+  ) {
+    this.llm = llm;
+    this.emitter = emitter;
+    this.personaInstructions = personaInstructions;
+    this.signal = signal;
+  }
+
+  /**
+   * Synthesizer agent node that combines information to answer the query
+   */
+  async execute(state: typeof AgentState.State): Promise<Command> {
+    try {
+      // Format the prompt using the external template
+      const template = PromptTemplate.fromTemplate(synthesizerPrompt);
+
+      const conversationHistory =
+        removeThinkingBlocksFromMessages(state.messages)
+          .map((msg) => `<${msg.getType()}>${msg.content}</${msg.getType()}>`)
+          .join('\n') || 'No previous conversation context';
+
+      const relevantDocuments = state.relevantDocuments
+        .map((doc, index) => {
+          const isFile = doc.metadata?.url?.toLowerCase().includes('file');
+          return `<${index + 1}>\n
+    <title>${doc.metadata.title}</title>
+    <source_type>${isFile ? 'file' : 'web'}</source_type>
+    ${isFile ? '' : '\n<url>' + doc.metadata.url + '</url>'}
+    <content>\n${doc.pageContent}\n    </content>
+</${index + 1}>`;
+        })
+        .join('\n');
+
+      const recursionLimitMessage = state.recursionLimitReached
+        ? `# ⚠️ IMPORTANT NOTICE - LIMITED INFORMATION
+**The search process was interrupted due to complexity limits. You MUST start your response with a warning about incomplete information and qualify all statements appropriately.**
+## ⚠️ CRITICAL: Incomplete Information Response Requirements
+**You MUST:**
+1. **Start your response** with a clear warning that the information may be incomplete or conflicting
+2. **Acknowledge limitations** throughout your response where information gaps exist
+3. **Be transparent** about what you cannot determine from the available sources
+4. **Suggest follow-up actions** for the user to get more complete information
+5. **Qualify your statements** with phrases like "based on available information" or "from the limited sources gathered"
+
+**Example opening for incomplete information responses:**
+"⚠️ **Please note:** This response is based on incomplete information due to search complexity limits. The findings below may be missing important details or conflicting perspectives. I recommend verifying this information through additional research or rephrasing your query for better results.
+
+`
+        : '';
+
+      // If we have limited documents due to recursion limit, acknowledge this
+      const documentsAvailable = state.relevantDocuments?.length || 0;
+      const limitedInfoNote =
+        state.recursionLimitReached && documentsAvailable === 0
+          ? '**CRITICAL: No source documents were gathered due to search limitations.**\n\n'
+          : state.recursionLimitReached
+            ? `**NOTICE: Search was interrupted with ${documentsAvailable} documents gathered.**\n\n`
+            : '';
+
+      const formattedPrompt = await template.format({
+        personaInstructions: this.personaInstructions,
+        conversationHistory: conversationHistory,
+        relevantDocuments: relevantDocuments,
+        query: state.originalQuery || state.query,
+        recursionLimitReached: recursionLimitMessage + limitedInfoNote,
+      });
+
+      // Stream the response in real-time using LLM streaming capabilities
+      let fullResponse = '';
+
+      // Emit the sources as a data response
+      this.emitter.emit(
+        'data',
+        JSON.stringify({
+          type: 'sources',
+          data: state.relevantDocuments,
+          searchQuery: '',
+          searchUrl: '',
+        }),
+      );
+
+      const stream = await this.llm.stream(
+        [
+          new SystemMessage(formattedPrompt),
+          new HumanMessage(state.originalQuery || state.query),
+        ],
+        { signal: this.signal },
+      );
+
+      for await (const chunk of stream) {
+        if (this.signal.aborted) {
+          break;
+        }
+
+        const content = chunk.content;
+        if (typeof content === 'string' && content.length > 0) {
+          fullResponse += content;
+
+          // Emit each chunk as a data response in real-time
+          this.emitter.emit(
+            'data',
+            JSON.stringify({
+              type: 'response',
+              data: content,
+            }),
+          );
+        }
+      }
+
+      // Emit model stats and end signal after streaming is complete
+      const modelName = getModelName(this.llm);
+      this.emitter.emit(
+        'stats',
+        JSON.stringify({
+          type: 'modelStats',
+          data: { modelName },
+        }),
+      );
+
+      this.emitter.emit('end');
+
+      // Create the final response message with the complete content
+      const response = new SystemMessage(fullResponse);
+
+      return new Command({
+        goto: END,
+        update: {
+          messages: [response],
+        },
+      });
+    } catch (error) {
+      console.error('Synthesis error:', error);
+      const errorMessage = new SystemMessage(
+        `Failed to synthesize answer: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+
+      return new Command({
+        goto: END,
+        update: {
+          messages: [errorMessage],
+        },
+      });
+    }
+  }
+}
--- a/src/lib/agents/taskManagerAgent.ts
+++ b/src/lib/agents/taskManagerAgent.ts
@ -0,0 +1,225 @@
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import { AIMessage } from '@langchain/core/messages';
+import { PromptTemplate } from '@langchain/core/prompts';
+import { Command } from '@langchain/langgraph';
+import { EventEmitter } from 'events';
+import { z } from 'zod';
+import { taskBreakdownPrompt } from '../prompts/taskBreakdown';
+import { AgentState } from './agentState';
+import { setTemperature } from '../utils/modelUtils';
+import { withStructuredOutput } from '../utils/structuredOutput';
+
+// Define Zod schema for structured task breakdown output
+const TaskBreakdownSchema = z.object({
+  tasks: z
+    .array(z.string())
+    .describe(
+      'Array of specific, focused tasks broken down from the original query',
+    ),
+  reasoning: z
+    .string()
+    .describe(
+      'Explanation of how and why the query was broken down into these tasks',
+    ),
+});
+
+type TaskBreakdown = z.infer<typeof TaskBreakdownSchema>;
+
+export class TaskManagerAgent {
+  private llm: BaseChatModel;
+  private emitter: EventEmitter;
+  private systemInstructions: string;
+  private signal: AbortSignal;
+
+  constructor(
+    llm: BaseChatModel,
+    emitter: EventEmitter,
+    systemInstructions: string,
+    signal: AbortSignal,
+  ) {
+    this.llm = llm;
+    this.emitter = emitter;
+    this.systemInstructions = systemInstructions;
+    this.signal = signal;
+  }
+
+  /**
+   * Task manager agent node - breaks down complex questions into smaller tasks
+   */
+  async execute(state: typeof AgentState.State): Promise<Command> {
+    try {
+      //setTemperature(this.llm, 0); // Set temperature to 0 for deterministic output
+
+      // Check if we're in task progression mode (tasks already exist and we're processing them)
+      if (state.tasks && state.tasks.length > 0) {
+        const currentTaskIndex = state.currentTaskIndex || 0;
+        const hasMoreTasks = currentTaskIndex < state.tasks.length - 1;
+
+        if (hasMoreTasks) {
+          // Move to next task
+          const nextTaskIndex = currentTaskIndex + 1;
+          this.emitter.emit('agent_action', {
+            type: 'agent_action',
+            data: {
+              action: 'PROCEEDING_TO_NEXT_TASK',
+              message: `Task ${currentTaskIndex + 1} completed. Moving to task ${nextTaskIndex + 1} of ${state.tasks.length}.`,
+              details: {
+                completedTask: state.tasks[currentTaskIndex],
+                nextTask: state.tasks[nextTaskIndex],
+                taskIndex: nextTaskIndex + 1,
+                totalTasks: state.tasks.length,
+                documentCount: state.relevantDocuments.length,
+                query: state.originalQuery || state.query,
+              },
+            },
+          });
+
+          return new Command({
+            goto: 'content_router',
+            update: {
+              // messages: [
+              //   new AIMessage(
+              //     `Task ${currentTaskIndex + 1} completed. Processing task ${nextTaskIndex + 1} of ${state.tasks.length}: "${state.tasks[nextTaskIndex]}"`,
+              //   ),
+              // ],
+              currentTaskIndex: nextTaskIndex,
+            },
+          });
+        } else {
+          // All tasks completed, move to analysis
+          this.emitter.emit('agent_action', {
+            type: 'agent_action',
+            data: {
+              action: 'ALL_TASKS_COMPLETED',
+              message: `All ${state.tasks.length} tasks completed. Ready for analysis.`,
+              details: {
+                totalTasks: state.tasks.length,
+                documentCount: state.relevantDocuments.length,
+                query: state.originalQuery || state.query,
+              },
+            },
+          });
+
+          return new Command({
+            goto: 'analyzer',
+            // update: {
+            //   messages: [
+            //     new AIMessage(
+            //       `All ${state.tasks.length} tasks completed. Moving to analysis phase.`,
+            //     ),
+            //   ],
+            // },
+          });
+        }
+      }
+
+      // Original task breakdown logic for new queries
+      // Emit task analysis event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'ANALYZING_TASK_COMPLEXITY',
+          message: `Analyzing question to determine if it needs to be broken down into smaller tasks`,
+          details: {
+            query: state.query,
+            currentTasks: state.tasks?.length || 0,
+          },
+        },
+      });
+
+      const template = PromptTemplate.fromTemplate(taskBreakdownPrompt);
+
+      // Create file context information
+      const fileContext =
+        state.fileIds && state.fileIds.length > 0
+          ? `Files attached: ${state.fileIds.length} file(s) are available for analysis. Consider creating tasks that can leverage these attached files when appropriate.`
+          : 'No files attached: Focus on tasks that can be answered through web research or general knowledge.';
+
+      const prompt = await template.format({
+        systemInstructions: this.systemInstructions,
+        fileContext: fileContext,
+        query: state.query,
+      });
+
+      // Use structured output for task breakdown
+      const structuredLlm = withStructuredOutput(
+        this.llm,
+        TaskBreakdownSchema,
+        {
+          name: 'break_down_tasks',
+        },
+      );
+
+      const taskBreakdownResult = (await structuredLlm.invoke([prompt], {
+        signal: this.signal,
+      })) as TaskBreakdown;
+
+      console.log('Task breakdown response:', taskBreakdownResult);
+
+      // Extract tasks from structured response
+      const taskLines = taskBreakdownResult.tasks.filter(
+        (task) => task.trim().length > 0,
+      );
+
+      if (taskLines.length === 0) {
+        // Fallback: if no tasks found, use the original query
+        taskLines.push(state.query);
+      }
+
+      console.log(
+        `Task breakdown completed: ${taskLines.length} tasks identified`,
+      );
+      console.log('Reasoning:', taskBreakdownResult.reasoning);
+      taskLines.forEach((task, index) => {
+        console.log(`Task ${index + 1}: ${task}`);
+      });
+
+      // Emit task breakdown completion event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'TASK_BREAKDOWN_COMPLETED',
+          message: `Question broken down into ${taskLines.length} focused ${taskLines.length === 1 ? 'task' : 'tasks'}`,
+          details: {
+            query: state.query,
+            taskCount: taskLines.length,
+            tasks: taskLines,
+            reasoning: taskBreakdownResult.reasoning,
+          },
+        },
+      });
+
+      const responseMessage =
+        taskLines.length === 1
+          ? 'Question is already focused and ready for processing'
+          : `Question broken down into ${taskLines.length} focused tasks for parallel processing`;
+
+      return new Command({
+        goto: 'content_router', // Route to content router to decide between file search, web search, or analysis
+        update: {
+          // messages: [new AIMessage(responseMessage)],
+          tasks: taskLines,
+          currentTaskIndex: 0,
+          originalQuery: state.originalQuery || state.query, // Preserve original if not already set
+        },
+      });
+    } catch (error) {
+      console.error('Task breakdown error:', error);
+      const errorMessage = new AIMessage(
+        `Task breakdown failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+
+      return new Command({
+        goto: 'content_router', // Fallback to content router with original query
+        update: {
+          messages: [errorMessage],
+          tasks: [state.query], // Use original query as single task
+          currentTaskIndex: 0,
+          originalQuery: state.originalQuery || state.query, // Preserve original if not already set
+        },
+      });
+    } finally {
+      setTemperature(this.llm, undefined); // Reset temperature to default
+    }
+  }
+}
--- a/src/lib/agents/urlSummarizationAgent.ts
+++ b/src/lib/agents/urlSummarizationAgent.ts
@ -0,0 +1,300 @@
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import { AIMessage } from '@langchain/core/messages';
+import { Command, END } from '@langchain/langgraph';
+import { EventEmitter } from 'events';
+import { Document } from 'langchain/document';
+import { AgentState } from './agentState';
+import { getWebContent } from '../utils/documents';
+import { removeThinkingBlocks } from '../utils/contentUtils';
+import { setTemperature } from '../utils/modelUtils';
+
+export class URLSummarizationAgent {
+  private llm: BaseChatModel;
+  private emitter: EventEmitter;
+  private systemInstructions: string;
+  private signal: AbortSignal;
+
+  constructor(
+    llm: BaseChatModel,
+    emitter: EventEmitter,
+    systemInstructions: string,
+    signal: AbortSignal,
+  ) {
+    this.llm = llm;
+    this.emitter = emitter;
+    this.systemInstructions = systemInstructions;
+    this.signal = signal;
+  }
+
+  /**
+   * URL processing agent node
+   */
+  async execute(state: typeof AgentState.State): Promise<Command> {
+    try {
+      setTemperature(this.llm, 0); // Set temperature to 0 for deterministic output
+
+      // Use pre-analyzed URLs from ContentRouterAgent
+      const urlsToProcess = state.urlsToSummarize || [];
+      const summarizationIntent =
+        state.summarizationIntent ||
+        'process content to help answer the user query';
+
+      if (urlsToProcess.length === 0) {
+        console.log(
+          'No URLs found for processing, routing back to content router',
+        );
+        return new Command({
+          goto: 'content_router',
+          // update: {
+          //   messages: [
+          //     new AIMessage(
+          //       'No URLs found for processing, routing to content router',
+          //     ),
+          //   ],
+          // },
+        });
+      }
+
+      console.log(`URL processing detected. URLs: ${urlsToProcess.join(', ')}`);
+      console.log(`Processing intent: ${summarizationIntent}`);
+
+      // Emit URL detection event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'URL_PROCESSING_DETECTED',
+          message: `Processing ${urlsToProcess.length} URL(s) to extract content for analysis`,
+          details: {
+            query: state.query,
+            urls: urlsToProcess,
+            intent: summarizationIntent,
+          },
+        },
+      });
+
+      const documents: Document[] = [];
+
+      // Process each URL
+      for (const url of urlsToProcess) {
+        if (this.signal.aborted) {
+          console.warn('URL summarization operation aborted by signal');
+          break;
+        }
+
+        try {
+          // Emit URL processing event
+          this.emitter.emit('agent_action', {
+            type: 'agent_action',
+            data: {
+              action: 'PROCESSING_URL',
+              message: `Retrieving and processing content from: ${url}`,
+              details: {
+                query: state.query,
+                sourceUrl: url,
+                intent: summarizationIntent,
+              },
+            },
+          });
+
+          // Fetch full content using the enhanced web content retrieval
+          const webContent = await getWebContent(url, true);
+
+          if (!webContent || !webContent.pageContent) {
+            console.warn(`No content retrieved from URL: ${url}`);
+
+            // Emit URL processing failure event
+            this.emitter.emit('agent_action', {
+              type: 'agent_action',
+              data: {
+                action: 'URL_PROCESSING_FAILED',
+                message: `Failed to retrieve content from: ${url}`,
+                details: {
+                  query: state.query,
+                  sourceUrl: url,
+                  reason: 'No content retrieved',
+                },
+              },
+            });
+            continue;
+          }
+
+          const contentLength = webContent.pageContent.length;
+          let finalContent: string;
+          let processingType: string;
+
+          // If content is short (< 4000 chars), use it directly; otherwise summarize
+          if (contentLength < 4000) {
+            finalContent = webContent.pageContent;
+            processingType = 'url-direct-content';
+
+            console.log(
+              `Content is short (${contentLength} chars), using directly without summarization`,
+            );
+
+            // Emit direct content usage event
+            this.emitter.emit('agent_action', {
+              type: 'agent_action',
+              data: {
+                action: 'URL_DIRECT_CONTENT',
+                message: `Content is short (${contentLength} chars), using directly from: ${url}`,
+                details: {
+                  query: state.query,
+                  sourceUrl: url,
+                  sourceTitle: webContent.metadata.title || 'Web Page',
+                  contentLength: contentLength,
+                  intent: summarizationIntent,
+                },
+              },
+            });
+          } else {
+            // Content is long, summarize using LLM
+            console.log(
+              `Content is long (${contentLength} chars), generating summary`,
+            );
+
+            const systemPrompt = this.systemInstructions
+              ? `${this.systemInstructions}\n\n`
+              : '';
+
+            const summarizationPrompt = `${systemPrompt}You are a web content processor. Extract and summarize ONLY the information from the provided web page content that is relevant to the user's query.
+
+# Critical Instructions
+- Output ONLY a summary of the web page content provided below
+- Focus on information that relates to or helps answer the user's query
+- Do NOT add pleasantries, greetings, or conversational elements
+- Do NOT mention missing URLs, other pages, or content not provided
+- Do NOT ask follow-up questions or suggest additional actions
+- Do NOT add commentary about the user's request or query
+- Present the information in a clear, well-structured format with key facts and details
+- Include all relevant details that could help answer the user's question
+
+# User's Query: ${state.query}
+
+# Content Title: ${webContent.metadata.title || 'Web Page'}
+# Content URL: ${url}
+
+# Web Page Content to Summarize:
+${webContent.pageContent}
+
+Provide a comprehensive summary of the above web page content, focusing on information relevant to the user's query:`;
+
+            const result = await this.llm.invoke(summarizationPrompt, {
+              signal: this.signal,
+            });
+
+            finalContent = removeThinkingBlocks(result.content as string);
+            processingType = 'url-content-extraction';
+          }
+
+          if (finalContent && finalContent.trim().length > 0) {
+            const document = new Document({
+              pageContent: finalContent,
+              metadata: {
+                title: webContent.metadata.title || 'URL Content',
+                url: url,
+                source: url,
+                processingType: processingType,
+                processingIntent: summarizationIntent,
+                originalContentLength: contentLength,
+              },
+            });
+
+            documents.push(document);
+
+            // Emit successful URL processing event
+            this.emitter.emit('agent_action', {
+              type: 'agent_action',
+              data: {
+                action: 'URL_CONTENT_EXTRACTED',
+                message: `Successfully processed content from: ${url}`,
+                details: {
+                  query: state.query,
+                  sourceUrl: url,
+                  sourceTitle: webContent.metadata.title || 'Web Page',
+                  contentLength: finalContent.length,
+                  originalContentLength: contentLength,
+                  processingType: processingType,
+                  intent: summarizationIntent,
+                },
+              },
+            });
+
+            console.log(
+              `Successfully processed content from ${url} (${finalContent.length} characters, ${processingType})`,
+            );
+          } else {
+            console.warn(`No valid content generated for URL: ${url}`);
+          }
+        } catch (error) {
+          console.error(`Error processing URL ${url}:`, error);
+
+          // Emit URL processing error event
+          this.emitter.emit('agent_action', {
+            type: 'agent_action',
+            data: {
+              action: 'URL_PROCESSING_ERROR',
+              message: `Error processing URL: ${url}`,
+              details: {
+                query: state.query,
+                sourceUrl: url,
+                error: error instanceof Error ? error.message : 'Unknown error',
+              },
+            },
+          });
+        }
+      }
+
+      if (documents.length === 0) {
+        const errorMessage = `No content could be retrieved or summarized from the provided URL(s): ${urlsToProcess.join(', ')}`;
+        console.error(errorMessage);
+
+        return new Command({
+          goto: 'analyzer',
+          // update: {
+          //   messages: [new AIMessage(errorMessage)],
+          // },
+        });
+      }
+
+      // Emit completion event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'URL_PROCESSING_COMPLETED',
+          message: `Successfully processed ${documents.length} URL(s) and extracted content`,
+          details: {
+            query: state.query,
+            processedUrls: urlsToProcess.length,
+            successfulExtractions: documents.length,
+            intent: summarizationIntent,
+          },
+        },
+      });
+
+      const responseMessage = `URL processing completed. Successfully processed ${documents.length} out of ${urlsToProcess.length} URLs.`;
+      console.log(responseMessage);
+
+      return new Command({
+        goto: 'analyzer', // Route to analyzer to continue with normal workflow after URL processing
+        update: {
+          // messages: [new AIMessage(responseMessage)],
+          relevantDocuments: documents,
+        },
+      });
+    } catch (error) {
+      console.error('URL summarization error:', error);
+      const errorMessage = new AIMessage(
+        `URL summarization failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+
+      return new Command({
+        goto: END,
+        update: {
+          messages: [errorMessage],
+        },
+      });
+    } finally {
+      setTemperature(this.llm, undefined); // Reset temperature to default
+    }
+  }
+}
--- a/src/lib/agents/webSearchAgent.ts
+++ b/src/lib/agents/webSearchAgent.ts
@ -0,0 +1,461 @@
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import { AIMessage } from '@langchain/core/messages';
+import { PromptTemplate } from '@langchain/core/prompts';
+import { Command, END } from '@langchain/langgraph';
+import { EventEmitter } from 'events';
+import { Document } from 'langchain/document';
+import { z } from 'zod';
+import LineOutputParser from '../outputParsers/lineOutputParser';
+import { webSearchRetrieverAgentPrompt } from '../prompts/webSearch';
+import { searchSearxng } from '../searxng';
+import { formatDateForLLM } from '../utils';
+import { summarizeWebContent } from '../utils/summarizeWebContent';
+import {
+  analyzePreviewContent,
+  PreviewContent,
+} from '../utils/analyzePreviewContent';
+import { AgentState } from './agentState';
+import { setTemperature } from '../utils/modelUtils';
+import { Embeddings } from '@langchain/core/embeddings';
+import { removeThinkingBlocksFromMessages } from '../utils/contentUtils';
+import computeSimilarity from '../utils/computeSimilarity';
+import { withStructuredOutput } from '../utils/structuredOutput';
+
+// Define Zod schema for structured search query output
+const SearchQuerySchema = z.object({
+  searchQuery: z
+    .string()
+    .describe('The optimized search query to use for web search'),
+  reasoning: z
+    .string()
+    .describe(
+      'Explanation of how the search query was optimized for better results',
+    ),
+});
+
+type SearchQuery = z.infer<typeof SearchQuerySchema>;
+
+export class WebSearchAgent {
+  private llm: BaseChatModel;
+  private emitter: EventEmitter;
+  private systemInstructions: string;
+  private signal: AbortSignal;
+  private embeddings: Embeddings;
+
+  constructor(
+    llm: BaseChatModel,
+    emitter: EventEmitter,
+    systemInstructions: string,
+    signal: AbortSignal,
+    embeddings: Embeddings,
+  ) {
+    this.llm = llm;
+    this.emitter = emitter;
+    this.systemInstructions = systemInstructions;
+    this.signal = signal;
+    this.embeddings = embeddings;
+  }
+
+  /**
+   * Web search agent node
+   */
+  async execute(state: typeof AgentState.State): Promise<Command> {
+    try {
+      //setTemperature(this.llm, 0); // Set temperature to 0 for deterministic output
+
+      // Determine current task to process
+      const currentTask =
+        state.tasks && state.tasks.length > 0
+          ? state.tasks[state.currentTaskIndex || 0]
+          : state.query;
+
+      console.log(
+        `Processing task ${(state.currentTaskIndex || 0) + 1} of ${state.tasks?.length || 1}: "${currentTask}"`,
+      );
+
+      // Emit preparing web search event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'PREPARING_SEARCH_QUERY',
+          // message: `Preparing search query`,
+          details: {
+            query: state.query,
+            currentTask: currentTask,
+            taskIndex: (state.currentTaskIndex || 0) + 1,
+            totalTasks: state.tasks?.length || 1,
+            searchInstructions: state.searchInstructions || currentTask,
+            documentCount: state.relevantDocuments.length,
+            searchIterations: state.searchInstructionHistory.length,
+          },
+        },
+      });
+
+      const template = PromptTemplate.fromTemplate(
+        webSearchRetrieverAgentPrompt,
+      );
+      const prompt = await template.format({
+        systemInstructions: this.systemInstructions,
+        query: currentTask, // Use current task instead of main query
+        date: formatDateForLLM(new Date()),
+        supervisor: state.searchInstructions,
+      });
+
+      // Use structured output for search query generation
+      const structuredLlm = withStructuredOutput(this.llm, SearchQuerySchema, {
+        name: 'generate_search_query',
+      });
+
+      const searchQueryResult = await structuredLlm.invoke(
+        [...removeThinkingBlocksFromMessages(state.messages), prompt],
+        { signal: this.signal },
+      );
+
+      const searchQuery = searchQueryResult.searchQuery;
+
+      console.log(`Performing web search for query: "${searchQuery}"`);
+      console.log('Search query reasoning:', searchQueryResult.reasoning);
+
+      // Emit executing web search event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'EXECUTING_WEB_SEARCH',
+          // message: `Searching the web for: '${searchQuery}'`,
+          details: {
+            query: state.query,
+            currentTask: currentTask,
+            taskIndex: (state.currentTaskIndex || 0) + 1,
+            totalTasks: state.tasks?.length || 1,
+            searchQuery: searchQuery,
+            documentCount: state.relevantDocuments.length,
+            searchIterations: state.searchInstructionHistory.length,
+          },
+        },
+      });
+
+      const searchResults = await searchSearxng(searchQuery, {
+        language: 'en',
+        engines: [],
+      });
+
+      // Emit web sources identified event
+      this.emitter.emit('agent_action', {
+        type: 'agent_action',
+        data: {
+          action: 'WEB_SOURCES_IDENTIFIED',
+          message: `Found ${searchResults.results.length} potential web sources`,
+          details: {
+            query: state.query,
+            currentTask: currentTask,
+            taskIndex: (state.currentTaskIndex || 0) + 1,
+            totalTasks: state.tasks?.length || 1,
+            searchQuery: searchQuery,
+            sourcesFound: searchResults.results.length,
+            documentCount: state.relevantDocuments.length,
+            searchIterations: state.searchInstructionHistory.length,
+          },
+        },
+      });
+
+      let bannedSummaryUrls = state.bannedSummaryUrls || [];
+      let bannedPreviewUrls = state.bannedPreviewUrls || [];
+      const queryVector = await this.embeddings.embedQuery(
+        state.originalQuery + ' ' + currentTask,
+      );
+
+      // Filter out banned URLs first
+      const filteredResults = searchResults.results.filter(
+        (result) =>
+          !bannedSummaryUrls.includes(result.url) &&
+          !bannedPreviewUrls.includes(result.url),
+      );
+
+      // Calculate similarities for all filtered results
+      const resultsWithSimilarity = await Promise.all(
+        filteredResults.map(async (result) => {
+          const vector = await this.embeddings.embedQuery(
+            result.title + ' ' + result.content || '',
+          );
+          const similarity = computeSimilarity(vector, queryVector);
+          return { result, similarity };
+        }),
+      );
+
+      let previewContents: PreviewContent[] = [];
+      // Always take the top 3 results for preview content
+      previewContents.push(
+        ...filteredResults.slice(0, 3).map((result) => ({
+          title: result.title || 'Untitled',
+          snippet: result.content || '',
+          url: result.url,
+        })),
+      );
+
+      // Sort by relevance score and take top 12 results for a total of 15
+      previewContents.push(
+        ...resultsWithSimilarity
+          .slice(3)
+          .sort((a, b) => b.similarity - a.similarity)
+          .slice(0, 12)
+          .map(({ result }) => ({
+            title: result.title || 'Untitled',
+            snippet: result.content || '',
+            url: result.url,
+          })),
+      );
+
+      console.log(
+        `Extracted preview content from ${previewContents.length} search results for analysis`,
+      );
+
+      // Perform preview analysis to determine if full content retrieval is needed
+      let previewAnalysisResult = null;
+      if (previewContents.length > 0) {
+        console.log(
+          'Starting preview content analysis to determine if full processing is needed',
+        );
+
+        // Emit preview analysis event
+        this.emitter.emit('agent_action', {
+          type: 'agent_action',
+          data: {
+            action: 'ANALYZING_PREVIEW_CONTENT',
+            message: `Analyzing ${previewContents.length} search result previews to determine processing approach`,
+            details: {
+              query: currentTask,
+              previewCount: previewContents.length,
+              documentCount: state.relevantDocuments.length,
+              searchIterations: state.searchInstructionHistory.length,
+            },
+          },
+        });
+
+        previewAnalysisResult = await analyzePreviewContent(
+          previewContents,
+          state.query,
+          currentTask,
+          removeThinkingBlocksFromMessages(state.messages),
+          this.llm,
+          this.systemInstructions,
+          this.signal,
+        );
+
+        console.log(
+          `Preview analysis result: ${previewAnalysisResult.isSufficient ? 'SUFFICIENT' : 'INSUFFICIENT'}${previewAnalysisResult.reason ? ` - ${previewAnalysisResult.reason}` : ''}`,
+        );
+      }
+
+      let documents: Document[] = [];
+      let attemptedUrlCount = 0; // Declare outside conditional blocks
+
+      // Conditional workflow based on preview analysis result
+      if (previewAnalysisResult && previewAnalysisResult.isSufficient) {
+        // Preview content is sufficient - create documents from preview content
+        console.log(
+          'Preview content determined sufficient - skipping full content retrieval',
+        );
+
+        // Emit preview processing event
+        this.emitter.emit('agent_action', {
+          type: 'agent_action',
+          data: {
+            action: 'PROCESSING_PREVIEW_CONTENT',
+            message: `Using preview content from ${previewContents.length} sources - no full content retrieval needed`,
+            details: {
+              query: currentTask,
+              previewCount: previewContents.length,
+              documentCount: state.relevantDocuments.length,
+              searchIterations: state.searchInstructionHistory.length,
+              processingType: 'preview-only',
+            },
+          },
+        });
+
+        // Create documents from preview content
+        documents = previewContents.map(
+          (content, index) =>
+            new Document({
+              pageContent: `# ${content.title}\n\n${content.snippet}`,
+              metadata: {
+                title: content.title,
+                url: content.url,
+                source: content.url,
+                processingType: 'preview-only',
+                snippet: content.snippet,
+              },
+            }),
+        );
+
+        previewContents.forEach((content) => {
+          bannedPreviewUrls.push(content.url); // Add to banned preview URLs to avoid duplicates
+        });
+
+        console.log(
+          `Created ${documents.length} documents from preview content`,
+        );
+      } else {
+        // Preview content is insufficient - proceed with full content processing
+        const insufficiencyReason =
+          previewAnalysisResult?.reason ||
+          'Preview content not available or insufficient';
+        console.log(
+          `Preview content insufficient: ${insufficiencyReason} - proceeding with full content retrieval`,
+        );
+
+        // Emit full processing event
+        this.emitter.emit('agent_action', {
+          type: 'agent_action',
+          data: {
+            action: 'PROCEEDING_WITH_FULL_ANALYSIS',
+            message: `Preview content insufficient - proceeding with detailed content analysis`,
+            details: {
+              query: currentTask,
+              insufficiencyReason: insufficiencyReason,
+              documentCount: state.relevantDocuments.length,
+              searchIterations: state.searchInstructionHistory.length,
+              processingType: 'full-content',
+            },
+          },
+        });
+
+        // Summarize the top 2 search results
+        for (const result of previewContents) {
+          if (this.signal.aborted) {
+            console.warn('Search operation aborted by signal');
+            break; // Exit if the operation is aborted
+          }
+
+          if (bannedSummaryUrls.includes(result.url)) {
+            console.log(`Skipping banned URL: ${result.url}`);
+            // Note: We don't emit an agent_action event for banned URLs as this is an internal
+            // optimization that should be transparent to the user
+            continue; // Skip banned URLs
+          }
+          // if (attemptedUrlCount >= 5) {
+          //   console.warn(
+          //     'Too many attempts to summarize URLs, stopping further attempts.',
+          //   );
+          //   break; // Limit the number of attempts to summarize URLs
+          // }
+          attemptedUrlCount++;
+
+          bannedSummaryUrls.push(result.url); // Add to banned URLs to avoid duplicates
+
+          if (documents.length >= 2) {
+            break; // Limit to top 1 document
+          }
+
+          // Emit analyzing source event
+          this.emitter.emit('agent_action', {
+            type: 'agent_action',
+            data: {
+              action: 'ANALYZING_SOURCE',
+              message: `Analyzing and summarizing content from: ${result.title || result.url}`,
+              details: {
+                query: currentTask,
+                sourceUrl: result.url,
+                sourceTitle: result.title || 'Untitled',
+                documentCount: state.relevantDocuments.length,
+                searchIterations: state.searchInstructionHistory.length,
+              },
+            },
+          });
+
+          const summaryResult = await summarizeWebContent(
+            result.url,
+            currentTask,
+            this.llm,
+            this.systemInstructions,
+            this.signal,
+          );
+
+          if (summaryResult.document) {
+            documents.push(summaryResult.document);
+
+            // Emit context updated event
+            this.emitter.emit('agent_action', {
+              type: 'agent_action',
+              data: {
+                action: 'CONTEXT_UPDATED',
+                message: `Added information from ${summaryResult.document.metadata.title || result.url} to context`,
+                details: {
+                  query: currentTask,
+                  sourceUrl: result.url,
+                  sourceTitle:
+                    summaryResult.document.metadata.title || 'Untitled',
+                  contentLength: summaryResult.document.pageContent.length,
+                  documentCount:
+                    state.relevantDocuments.length + documents.length,
+                  searchIterations: state.searchInstructionHistory.length,
+                },
+              },
+            });
+
+            console.log(
+              `Summarized content from ${result.url} to ${summaryResult.document.pageContent.length} characters. Content: ${summaryResult.document.pageContent}`,
+            );
+          } else {
+            console.warn(`No relevant content found for URL: ${result.url}`);
+
+            // Emit skipping irrelevant source event for non-relevant content
+            this.emitter.emit('agent_action', {
+              type: 'agent_action',
+              data: {
+                action: 'SKIPPING_IRRELEVANT_SOURCE',
+                message: `Source ${result.title || result.url} was not relevant - trying next`,
+                details: {
+                  query: state.query,
+                  sourceUrl: result.url,
+                  sourceTitle: result.title || 'Untitled',
+                  skipReason:
+                    summaryResult.notRelevantReason ||
+                    'Content was not relevant to the query',
+                  documentCount:
+                    state.relevantDocuments.length + documents.length,
+                  searchIterations: state.searchInstructionHistory.length,
+                },
+              },
+            });
+          }
+        }
+      } // Close the else block for full content processing
+
+      if (documents.length === 0) {
+        return new Command({
+          goto: 'analyzer',
+          // update: {
+          //   messages: [new AIMessage('No relevant documents found.')],
+          // },
+        });
+      }
+
+      const responseMessage = `Web search completed. ${documents.length === 0 && attemptedUrlCount < 5 ? 'This search query does not have enough relevant information. Try rephrasing your query or providing more context.' : `Found ${documents.length} results that are relevant to the query.`}`;
+      console.log(responseMessage);
+
+      return new Command({
+        goto: 'analyzer', // Route back to analyzer to process the results
+        update: {
+          // messages: [new AIMessage(responseMessage)],
+          relevantDocuments: documents,
+          bannedSummaryUrls: bannedSummaryUrls,
+          bannedPreviewUrls: bannedPreviewUrls,
+        },
+      });
+    } catch (error) {
+      console.error('Web search error:', error);
+      const errorMessage = new AIMessage(
+        `Web search failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
+      );
+
+      return new Command({
+        goto: END,
+        update: {
+          messages: [errorMessage],
+        },
+      });
+    } finally {
+      setTemperature(this.llm, undefined); // Reset temperature to default
+    }
+  }
+}
--- a/src/lib/cancel-tokens.ts
+++ b/src/lib/cancel-tokens.ts
@ -0,0 +1,36 @@
+export {};
+
+// In-memory map to store cancel tokens by messageId
+const cancelTokens: Record<string, AbortController> = {};
+
+// Register a cancel token for a message ID
+export function registerCancelToken(
+  messageId: string,
+  controller: AbortController,
+) {
+  cancelTokens[messageId] = controller;
+}
+
+// Remove a cancel token from the map
+export function cleanupCancelToken(messageId: string) {
+  var cancelled = false;
+  if (messageId in cancelTokens) {
+    delete cancelTokens[messageId];
+    cancelled = true;
+  }
+  return cancelled;
+}
+
+// Cancel a request by its message ID
+export function cancelRequest(messageId: string) {
+  const controller = cancelTokens[messageId];
+  if (controller) {
+    try {
+      controller.abort();
+    } catch (e) {
+      console.error(`Error aborting request for messageId ${messageId}:`, e);
+    }
+    return true;
+  }
+  return false;
+}
--- a/src/lib/chains/imageSearchAgent.ts
+++ b/src/lib/chains/imageSearchAgent.ts
@ -6,29 +6,74 @@ import {
 import { PromptTemplate } from '@langchain/core/prompts';
 import formatChatHistoryAsString from '../utils/formatHistory';
 import { BaseMessage } from '@langchain/core/messages';
-import { StringOutputParser } from '@langchain/core/output_parsers';
+import LineOutputParser from '../outputParsers/lineOutputParser';
 import { searchSearxng } from '../searxng';
+import { formatDateForLLM } from '../utils';
 import type { BaseChatModel } from '@langchain/core/language_models/chat_models';

 const imageSearchChainPrompt = `
-You will be given a conversation below and a follow up question. You need to rephrase the follow-up question so it is a standalone question that can be used by the LLM to search the web for images.
-You need to make sure the rephrased question agrees with the conversation and is relevant to the conversation.
+# Instructions
+- You will be given a question from a user and a conversation history
+- Rephrase the question based on the conversation so it is a standalone question that can be used to search for images that are relevant to the question
+- Ensure the rephrased question agrees with the conversation and is relevant to the conversation
+- If you are thinking or reasoning, use <think> tags to indicate your thought process
+- If you are thinking or reasoning, do not use <answer> and </answer> tags in your thinking. Those tags should only be used in the final output
+- Use the provided date to ensure the rephrased question is relevant to the current date and time if applicable

-Example:
-1. Follow up question: What is a cat?
-Rephrased: A cat
+# Data locations
+- The history is contained in the <conversation> tag after the <examples> below
+- The user question is contained in the <question> tag after the <examples> below
+- Output your answer in an <answer> tag
+- Current date is: {date}
+- Do not include any other text in your answer
  
-2. Follow up question: What is a car? How does it works?
-Rephrased: Car working
+<examples>
+## Example 1 input
+<conversation>
+  Who won the last F1 race?\nAyrton Senna won the Monaco Grand Prix. It was a tight race with lots of overtakes. Alain Prost was in the lead for most of the race until the last lap when Senna overtook them.
+</conversation>
+<question>
+  What were the highlights of the race?
+</question>

-3. Follow up question: How does an AC work?
-Rephrased: AC working
+## Example 1 output
+<answer>
+  F1 Monaco Grand Prix highlights
+</answer>

-Conversation:
+## Example 2 input
+<conversation>
+  What is the theory of relativity?
+</conversation>
+<question>
+  What is the theory of relativity?
+</question>
+
+## Example 2 output
+<answer>
+  Theory of relativity
+</answer>
+
+## Example 3 input
+<conversation>
+  I'm looking for a nice vacation spot. Where do you suggest?\nI suggest you go to Hawaii. It's a beautiful place with lots of beaches and activities to do.\nI love the beach! What are some activities I can do there?\nYou can go surfing, snorkeling, or just relax on the beach.
+</conversation>
+<question>
+  What are some activities I can do in Hawaii?
+</question>
+
+## Example 3 output
+<answer>
+  Hawaii activities
+</answer>
+</examples>
+
+<conversation>
 {chat_history}
-
-Follow up question: {query}
-Rephrased question:
+</conversation>
+<question>
+{query}
+</question>
 `;

 type ImageSearchChainInput = {
@ -42,9 +87,18 @@ interface ImageSearchResult {
  title: string;
 }

-const strParser = new StringOutputParser();
+const outputParser = new LineOutputParser({
+  key: 'answer',
+});
+
+const createImageSearchChain = (
+  llm: BaseChatModel,
+  systemInstructions?: string,
+) => {
+  const systemPrompt = systemInstructions ? `${systemInstructions}\n\n` : '';
+
+  const fullPrompt = `${systemPrompt}${imageSearchChainPrompt}`;

-const createImageSearchChain = (llm: BaseChatModel) => {
  return RunnableSequence.from([
    RunnableMap.from({
      chat_history: (input: ImageSearchChainInput) => {
@ -53,14 +107,13 @@ const createImageSearchChain = (llm: BaseChatModel) => {
      query: (input: ImageSearchChainInput) => {
        return input.query;
      },
+      date: () => formatDateForLLM(),
    }),
-    PromptTemplate.fromTemplate(imageSearchChainPrompt),
+    PromptTemplate.fromTemplate(fullPrompt),
    llm,
-    strParser,
-    RunnableLambda.from(async (input: string) => {
-      input = input.replace(/<think>.*?<\/think>/g, '');
-
-      const res = await searchSearxng(input, {
+    outputParser,
+    RunnableLambda.from(async (searchQuery: string) => {
+      const res = await searchSearxng(searchQuery, {
        engines: ['bing images', 'google images'],
      });

@ -76,7 +129,7 @@ const createImageSearchChain = (llm: BaseChatModel) => {
        }
      });

-      return images.slice(0, 10);
+      return images;
    }),
  ]);
 };
@ -84,8 +137,9 @@ const createImageSearchChain = (llm: BaseChatModel) => {
 const handleImageSearch = (
  input: ImageSearchChainInput,
  llm: BaseChatModel,
+  systemInstructions?: string,
 ) => {
-  const imageSearchChain = createImageSearchChain(llm);
+  const imageSearchChain = createImageSearchChain(llm, systemInstructions);
  return imageSearchChain.invoke(input);
 };

--- a/src/lib/chains/suggestionGeneratorAgent.ts
+++ b/src/lib/chains/suggestionGeneratorAgent.ts
@ -7,20 +7,34 @@ import { BaseChatModel } from '@langchain/core/language_models/chat_models';
 import { ChatOpenAI } from '@langchain/openai';

 const suggestionGeneratorPrompt = `
-You are an AI suggestion generator for an AI powered search engine. You will be given a conversation below. You need to generate 4-5 suggestions based on the conversation. The suggestion should be relevant to the conversation that can be used by the user to ask the chat model for more information.
-You need to make sure the suggestions are relevant to the conversation and are helpful to the user. Keep a note that the user might use these suggestions to ask a chat model for more information. 
-Make sure the suggestions are medium in length and are informative and relevant to the conversation.
+You are an AI suggestion generator for an AI powered search engine.

-Provide these suggestions separated by newlines between the XML tags <suggestions> and </suggestions>. For example:
+# Instructions
+- You will be given a conversation below
+- Generate 5 total suggestions based on the conversation
+  - Three of the suggestions should be relevant to the conversation so it can be used by the user to ask the chat model for more information
+  - Two of the suggestions should still be relevant to the conversation but could optionally steer the conversation in a different direction
+  - The suggestions should be in the form of questions
+  - The suggestions should not be something that is already in the conversation
+- The conversation history is provided in the conversation section below

+ # Output Format
+- If you are a thinking or reasoning AI, you should avoid using \`<suggestions>\` and \`</suggestions>\` tags in your thinking. Those tags should only be used in the final output.
+- Provide these suggestions separated by newlines between the XML tags <suggestions> and </suggestions>. For example:
+- Make sure each suggestion is a single line and does not contain any newlines or any formatting
+- Example output is provided in the example section below
+
+<example>
 <suggestions>
 Tell me more about SpaceX and their recent projects
 What is the latest news on SpaceX?
 Who is the CEO of SpaceX?
 </suggestions>
+</example>

-Conversation:
+<conversation>
 {chat_history}
+</conversation>
 `;

 type SuggestionGeneratorInput = {
@ -31,13 +45,20 @@ const outputParser = new ListLineOutputParser({
  key: 'suggestions',
 });

-const createSuggestionGeneratorChain = (llm: BaseChatModel) => {
+const createSuggestionGeneratorChain = (
+  llm: BaseChatModel,
+  systemInstructions?: string,
+) => {
+  const systemPrompt = systemInstructions ? `${systemInstructions}\n\n` : '';
+
+  const fullPrompt = `${systemPrompt}${suggestionGeneratorPrompt}`;
+
  return RunnableSequence.from([
    RunnableMap.from({
      chat_history: (input: SuggestionGeneratorInput) =>
        formatChatHistoryAsString(input.chat_history),
    }),
-    PromptTemplate.fromTemplate(suggestionGeneratorPrompt),
+    PromptTemplate.fromTemplate(fullPrompt),
    llm,
    outputParser,
  ]);
@ -46,9 +67,13 @@ const createSuggestionGeneratorChain = (llm: BaseChatModel) => {
 const generateSuggestions = (
  input: SuggestionGeneratorInput,
  llm: BaseChatModel,
+  systemInstructions?: string,
 ) => {
  (llm as unknown as ChatOpenAI).temperature = 0;
-  const suggestionGeneratorChain = createSuggestionGeneratorChain(llm);
+  const suggestionGeneratorChain = createSuggestionGeneratorChain(
+    llm,
+    systemInstructions,
+  );
  return suggestionGeneratorChain.invoke(input);
 };

--- a/src/lib/chains/videoSearchAgent.ts
+++ b/src/lib/chains/videoSearchAgent.ts
@ -6,30 +6,75 @@ import {
 import { PromptTemplate } from '@langchain/core/prompts';
 import formatChatHistoryAsString from '../utils/formatHistory';
 import { BaseMessage } from '@langchain/core/messages';
-import { StringOutputParser } from '@langchain/core/output_parsers';
+import LineOutputParser from '../outputParsers/lineOutputParser';
 import { searchSearxng } from '../searxng';
+import { formatDateForLLM } from '../utils';
 import type { BaseChatModel } from '@langchain/core/language_models/chat_models';

 const VideoSearchChainPrompt = `
-  You will be given a conversation below and a follow up question. You need to rephrase the follow-up question so it is a standalone question that can be used by the LLM to search Youtube for videos.
-  You need to make sure the rephrased question agrees with the conversation and is relevant to the conversation.
+# Instructions
+- You will be given a question from a user and a conversation history
+- Rephrase the question based on the conversation so it is a standalone question that can be used to search Youtube for videos
+- Ensure the rephrased question agrees with the conversation and is relevant to the conversation
+- If you are thinking or reasoning, use <think> tags to indicate your thought process
+- If you are thinking or reasoning, do not use <answer> and </answer> tags in your thinking. Those tags should only be used in the final output
+- Use the provided date to ensure the rephrased question is relevant to the current date and time if applicable

-  Example:
-  1. Follow up question: How does a car work?
-  Rephrased: How does a car work?
+# Data locations
+- The history is contained in the <conversation> tag after the <examples> below
+- The user question is contained in the <question> tag after the <examples> below
+- Output your answer in an <answer> tag
+- Current date is: {date}
+- Do not include any other text in your answer
  
-  2. Follow up question: What is the theory of relativity?
-  Rephrased: What is theory of relativity
+<examples>
+## Example 1 input
+<conversation>
+  Who won the last F1 race?\nAyrton Senna won the Monaco Grand Prix. It was a tight race with lots of overtakes. Alain Prost was in the lead for most of the race until the last lap when Senna overtook them.
+</conversation>
+<question>
+  What were the highlights of the race?
+</question>

-  3. Follow up question: How does an AC work?
-  Rephrased: How does an AC work
+## Example 1 output
+<answer>
+  F1 Monaco Grand Prix highlights
+</answer>

-  Conversation:
-  {chat_history}
+## Example 2 input
+<conversation>
+  What is the theory of relativity?
+</conversation>
+<question>
+  What is the theory of relativity?
+</question>

-  Follow up question: {query}
-  Rephrased question:
-  `;
+## Example 2 output
+<answer>
+  What is the theory of relativity?
+</answer>
+
+## Example 3 input
+<conversation>
+  I'm looking for a nice vacation spot. Where do you suggest?\nI suggest you go to Hawaii. It's a beautiful place with lots of beaches and activities to do.\nI love the beach! What are some activities I can do there?\nYou can go surfing, snorkeling, or just relax on the beach.
+</conversation>
+<question>
+  What are some activities I can do in Hawaii?
+</question>
+
+## Example 3 output
+<answer>
+  Activities to do in Hawaii
+</answer>
+</examples>
+
+<conversation>
+{chat_history}
+</conversation>
+<question>
+{query}
+</question>
+`;

 type VideoSearchChainInput = {
  chat_history: BaseMessage[];
@ -43,9 +88,18 @@ interface VideoSearchResult {
  iframe_src: string;
 }

-const strParser = new StringOutputParser();
+const answerParser = new LineOutputParser({
+  key: 'answer',
+});
+
+const createVideoSearchChain = (
+  llm: BaseChatModel,
+  systemInstructions?: string,
+) => {
+  const systemPrompt = systemInstructions ? `${systemInstructions}\n\n` : '';
+
+  const fullPrompt = `${systemPrompt}${VideoSearchChainPrompt}`;

-const createVideoSearchChain = (llm: BaseChatModel) => {
  return RunnableSequence.from([
    RunnableMap.from({
      chat_history: (input: VideoSearchChainInput) => {
@ -54,14 +108,13 @@ const createVideoSearchChain = (llm: BaseChatModel) => {
      query: (input: VideoSearchChainInput) => {
        return input.query;
      },
+      date: () => formatDateForLLM(),
    }),
-    PromptTemplate.fromTemplate(VideoSearchChainPrompt),
+    PromptTemplate.fromTemplate(fullPrompt),
    llm,
-    strParser,
-    RunnableLambda.from(async (input: string) => {
-      input = input.replace(/<think>.*?<\/think>/g, '');
-
-      const res = await searchSearxng(input, {
+    answerParser,
+    RunnableLambda.from(async (searchQuery: string) => {
+      const res = await searchSearxng(searchQuery, {
        engines: ['youtube'],
      });

@ -83,7 +136,7 @@ const createVideoSearchChain = (llm: BaseChatModel) => {
        }
      });

-      return videos.slice(0, 10);
+      return videos;
    }),
  ]);
 };
@ -91,8 +144,9 @@ const createVideoSearchChain = (llm: BaseChatModel) => {
 const handleVideoSearch = (
  input: VideoSearchChainInput,
  llm: BaseChatModel,
+  systemInstructions?: string,
 ) => {
-  const VideoSearchChain = createVideoSearchChain(llm);
+  const VideoSearchChain = createVideoSearchChain(llm, systemInstructions);
  return VideoSearchChain.invoke(input);
 };

--- a/src/lib/config.ts
+++ b/src/lib/config.ts
@ -15,6 +15,8 @@ interface Config {
  GENERAL: {
    SIMILARITY_MEASURE: string;
    KEEP_ALIVE: string;
+    BASE_URL?: string;
+    HIDDEN_MODELS: string[];
  };
  MODELS: {
    OPENAI: {
@ -41,6 +43,9 @@ interface Config {
    LM_STUDIO: {
      API_URL: string;
    };
+    OPENROUTER: {
+      API_KEY: string;
+    };
    CUSTOM_OPENAI: {
      API_URL: string;
      API_KEY: string;
@ -59,9 +64,41 @@ type RecursivePartial<T> = {
 const loadConfig = () => {
  // Server-side only
  if (typeof window === 'undefined') {
-    return toml.parse(
+    const config = toml.parse(
      fs.readFileSync(path.join(process.cwd(), `${configFileName}`), 'utf-8'),
    ) as any as Config;
+
+    // Ensure GENERAL section exists
+    if (!config.GENERAL) {
+      config.GENERAL = {} as any;
+    }
+
+    // Handle HIDDEN_MODELS - fix malformed table format to proper array
+    if (!config.GENERAL.HIDDEN_MODELS) {
+      config.GENERAL.HIDDEN_MODELS = [];
+    } else if (
+      typeof config.GENERAL.HIDDEN_MODELS === 'object' &&
+      !Array.isArray(config.GENERAL.HIDDEN_MODELS)
+    ) {
+      // Convert malformed table format to array
+      const hiddenModelsObj = config.GENERAL.HIDDEN_MODELS as any;
+      const hiddenModelsArray: string[] = [];
+
+      // Extract values from numeric keys and sort by key
+      const keys = Object.keys(hiddenModelsObj)
+        .map((k) => parseInt(k))
+        .filter((k) => !isNaN(k))
+        .sort((a, b) => a - b);
+      for (const key of keys) {
+        if (typeof hiddenModelsObj[key] === 'string') {
+          hiddenModelsArray.push(hiddenModelsObj[key]);
+        }
+      }
+
+      config.GENERAL.HIDDEN_MODELS = hiddenModelsArray;
+    }
+
+    return config;
  }

  // Client-side fallback - settings will be loaded via API
@ -73,10 +110,16 @@ export const getSimilarityMeasure = () =>

 export const getKeepAlive = () => loadConfig().GENERAL.KEEP_ALIVE;

+export const getBaseUrl = () => loadConfig().GENERAL.BASE_URL;
+
+export const getHiddenModels = () => loadConfig().GENERAL.HIDDEN_MODELS;
+
 export const getOpenaiApiKey = () => loadConfig().MODELS.OPENAI.API_KEY;

 export const getGroqApiKey = () => loadConfig().MODELS.GROQ.API_KEY;

+export const getOpenrouterApiKey = () => loadConfig().MODELS.OPENROUTER.API_KEY;
+
 export const getAnthropicApiKey = () => loadConfig().MODELS.ANTHROPIC.API_KEY;

 export const getGeminiApiKey = () => loadConfig().MODELS.GEMINI.API_KEY;
@ -111,17 +154,26 @@ const mergeConfigs = (current: any, update: any): any => {
    return update;
  }

+  // Handle arrays specifically - don't merge them, replace them
+  if (Array.isArray(update)) {
+    return update;
+  }
+
  const result = { ...current };

  for (const key in update) {
    if (Object.prototype.hasOwnProperty.call(update, key)) {
      const updateValue = update[key];

-      if (
+      // Handle arrays specifically - don't merge them, replace them
+      if (Array.isArray(updateValue)) {
+        result[key] = updateValue;
+      } else if (
        typeof updateValue === 'object' &&
        updateValue !== null &&
        typeof result[key] === 'object' &&
-        result[key] !== null
+        result[key] !== null &&
+        !Array.isArray(result[key])
      ) {
        result[key] = mergeConfigs(result[key], updateValue);
      } else if (updateValue !== undefined) {
--- a/src/lib/db/schema.ts
+++ b/src/lib/db/schema.ts
@ -17,6 +17,23 @@ interface File {
  fileId: string;
 }

+export const systemPrompts = sqliteTable('system_prompts', {
+  id: text('id')
+    .primaryKey()
+    .$defaultFn(() => crypto.randomUUID()),
+  name: text('name').notNull(),
+  content: text('content').notNull(),
+  type: text('type', { enum: ['system', 'persona'] })
+    .notNull()
+    .default('system'),
+  createdAt: integer('created_at', { mode: 'timestamp' })
+    .notNull()
+    .$defaultFn(() => new Date()),
+  updatedAt: integer('updated_at', { mode: 'timestamp' })
+    .notNull()
+    .$defaultFn(() => new Date()),
+});
+
 export const chats = sqliteTable('chats', {
  id: text('id').primaryKey(),
  title: text('title').notNull(),
--- a/src/lib/outputParsers/lineOutputParser.ts
+++ b/src/lib/outputParsers/lineOutputParser.ts
@ -1,4 +1,5 @@
 import { BaseOutputParser } from '@langchain/core/output_parsers';
+import { removeThinkingBlocks } from '../utils/contentUtils';

 interface LineOutputParserArgs {
  key?: string;
@ -21,6 +22,10 @@ class LineOutputParser extends BaseOutputParser<string> {
  async parse(text: string): Promise<string> {
    text = text.trim() || '';

+    // First, remove all <think>...</think> blocks to avoid parsing tags inside thinking content
+    // This might be a little aggressive. Prompt massaging might be all we need, but this is a guarantee and should rarely mess anything up.
+    text = removeThinkingBlocks(text);
+
    const regex = /^(\s*(-|\*|\d+\.\s|\d+\)\s|\u2022)\s*)+/;
    const startKeyIndex = text.indexOf(`<${this.key}>`);
    const endKeyIndex = text.indexOf(`</${this.key}>`);
--- a/src/lib/outputParsers/listLineOutputParser.ts
+++ b/src/lib/outputParsers/listLineOutputParser.ts
@ -1,4 +1,5 @@
 import { BaseOutputParser } from '@langchain/core/output_parsers';
+import { removeThinkingBlocks } from '../utils/contentUtils';

 interface LineListOutputParserArgs {
  key?: string;
@ -21,6 +22,10 @@ class LineListOutputParser extends BaseOutputParser<string[]> {
  async parse(text: string): Promise<string[]> {
    text = text.trim() || '';

+    // First, remove all <think>...</think> blocks to avoid parsing tags inside thinking content
+    // This might be a little aggressive. Prompt massaging might be all we need, but this is a guarantee and should rarely mess anything up.
+    text = removeThinkingBlocks(text);
+
    const regex = /^(\s*(-|\*|\d+\.\s|\d+\)\s|\u2022)\s*)+/;
    const startKeyIndex = text.indexOf(`<${this.key}>`);
    const endKeyIndex = text.indexOf(`</${this.key}>`);
--- a/src/lib/prompts/academicSearch.ts
+++ b/src/lib/prompts/academicSearch.ts
@ -1,69 +0,0 @@
-export const academicSearchRetrieverPrompt = `
-You will be given a conversation below and a follow up question. You need to rephrase the follow-up question if needed so it is a standalone question that can be used by the LLM to search the web for information.
-If it is a writing task or a simple hi, hello rather than a question, you need to return \`not_needed\` as the response.
-
-Example:
-1. Follow up question: How does stable diffusion work?
-Rephrased: Stable diffusion working
-
-2. Follow up question: What is linear algebra?
-Rephrased: Linear algebra
-
-3. Follow up question: What is the third law of thermodynamics?
-Rephrased: Third law of thermodynamics
-
-Conversation:
-{chat_history}
-
-Follow up question: {query}
-Rephrased question:
-`;
-
-export const academicSearchResponsePrompt = `
-   You are Perplexica, an AI model skilled in web search and crafting detailed, engaging, and well-structured answers. You excel at summarizing web pages and extracting relevant information to create professional, blog-style responses.
-
-    Your task is to provide answers that are:
-    - **Informative and relevant**: Thoroughly address the user's query using the given context.
-    - **Well-structured**: Include clear headings and subheadings, and use a professional tone to present information concisely and logically.
-    - **Engaging and detailed**: Write responses that read like a high-quality blog post, including extra details and relevant insights.
-    - **Cited and credible**: Use inline citations with [number] notation to refer to the context source(s) for each fact or detail included.
-    - **Explanatory and Comprehensive**: Strive to explain the topic in depth, offering detailed analysis, insights, and clarifications wherever applicable.
-
-    ### Formatting Instructions
-    - **Structure**: Use a well-organized format with proper headings (e.g., "## Example heading 1" or "## Example heading 2"). Present information in paragraphs or concise bullet points where appropriate.
-    - **Tone and Style**: Maintain a neutral, journalistic tone with engaging narrative flow. Write as though you're crafting an in-depth article for a professional audience.
-    - **Markdown Usage**: Format your response with Markdown for clarity. Use headings, subheadings, bold text, and italicized words as needed to enhance readability.
-    - **Length and Depth**: Provide comprehensive coverage of the topic. Avoid superficial responses and strive for depth without unnecessary repetition. Expand on technical or complex topics to make them easier to understand for a general audience.
-    - **No main heading/title**: Start your response directly with the introduction unless asked to provide a specific title.
-    - **Conclusion or Summary**: Include a concluding paragraph that synthesizes the provided information or suggests potential next steps, where appropriate.
-
-    ### Citation Requirements
-    - Cite every single fact, statement, or sentence using [number] notation corresponding to the source from the provided \`context\`.
-    - Integrate citations naturally at the end of sentences or clauses as appropriate. For example, "The Eiffel Tower is one of the most visited landmarks in the world[1]."
-    - Ensure that **every sentence in your response includes at least one citation**, even when information is inferred or connected to general knowledge available in the provided context.
-    - Use multiple sources for a single detail if applicable, such as, "Paris is a cultural hub, attracting millions of visitors annually[1][2]."
-    - Always prioritize credibility and accuracy by linking all statements back to their respective context sources.
-    - Avoid citing unsupported assumptions or personal interpretations; if no source supports a statement, clearly indicate the limitation.
-
-    ### Special Instructions
-    - If the query involves technical, historical, or complex topics, provide detailed background and explanatory sections to ensure clarity.
-    - If the user provides vague input or if relevant information is missing, explain what additional details might help refine the search.
-    - If no relevant information is found, say: "Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?" Be transparent about limitations and suggest alternatives or ways to reframe the query.
-    - You are set on focus mode 'Academic', this means you will be searching for academic papers and articles on the web.
-    
-    ### User instructions
-    These instructions are shared to you by the user and not by the system. You will have to follow them but give them less priority than the above instructions. If the user has provided specific instructions or preferences, incorporate them into your response while adhering to the overall guidelines.
-    {systemInstructions}
-
-    ### Example Output
-    - Begin with a brief introduction summarizing the event or query topic.
-    - Follow with detailed sections under clear headings, covering all aspects of the query if possible.
-    - Provide explanations or historical context as needed to enhance understanding.
-    - End with a conclusion or overall perspective if relevant.
-
-    <context>
-    {context}
-    </context>
-
-    Current date & time in ISO format (UTC timezone) is: {date}.
-`;
--- a/src/lib/prompts/analyzer.ts
+++ b/src/lib/prompts/analyzer.ts
@ -0,0 +1,182 @@
+export const decideNextActionPrompt = `You are an expert content analyzer.
+Your task is to analyze the provided content and determine if we have enough information to fully answer the user's query.
+
+# Instructions
+- Carefully analyze the content of the context provided **and** the historical content of the conversation to determine if it contains sufficient information to answer the user's query
+- The context may be empty, if the historical content is sufficient, you can still consider it sufficient
+- Historic content should generally be considered factual and does not require additional confirmation unless the user explicitly asks for confirmation or indicates that it was incorrect
+- If the user is asking for a specific number of sources and the context does not provide enough, consider the content insufficient
+
+# Source Type Awareness
+When analyzing the content, be aware that documents may come from different sources:
+- **File documents**: Content extracted from user-attached files (identified by metadata indicating file source)
+- **Web documents**: Content retrieved from web searches (identified by URLs and web source metadata)
+- **Chat history**: Previous messages in the conversation that may provide additional content
+- **Mixed sources**: The content may include a combination of file documents, web documents, and chat history
+
+Consider the following when evaluating sufficiency:
+- File documents may contain user-specific, proprietary, or contextual information that cannot be found elsewhere
+- Web documents provide current, general, and publicly available information
+- Chat history provides conversational context and may include user preferences, past interactions, or clarifications
+- The combination of these sources should be evaluated holistically to determine if they collectively provide enough information to answer the user's query
+- File content should be prioritized when answering questions specifically about attached documents
+
+# Response Options Decision Tree
+
+## Step 1: Check if content is sufficient
+- If provided content contains enough information to answer the user's query → respond with \`good_content\`
+- If the content fully answers the user's query with complete information → respond with \`good_content\`
+- If the user is requesting to use the existing content to answer their query → respond with \`good_content\`
+- If the user is requesting to avoid web searches → respond with \`good_content\`
+- If the user is asking you to be creative, such as writing a story, poem, or creative content → respond with \`good_content\` unless the content is clearly insufficient
+- If file documents contain complete information for file-specific queries → respond with \`good_content\`
+- If the user is requesting specific web content and there is a source that corresponds to that request in the context, it can be considered sufficient even if the content is not exhaustive or looks like errors → respond with \`good_content\`
+
+## Step 2: If content is insufficient, determine the type of missing information
+
+### Use \`need_user_info\` when the missing information is:
+**Personal/Subjective Information:**
+- User's personal preferences, opinions, or experiences
+- User's specific situation, location, or circumstances  
+- User's budget, timeline, or constraints
+- User's skill level, background, or expertise
+- User's goals, intentions, or desired outcomes
+- Configuration details about user's specific setup/environment
+- User's past experiences with products/services
+- User's access to specific resources or tools
+- Related to creative or subjective tasks
+
+**Context-Dependent Information:**
+- "What should I do in my specific situation?"
+- "What's best for me personally?"
+- "How do I configure my specific system?"
+- "What happened in my case?"
+
+**Examples requiring user info:**
+- "What laptop should I buy?" (missing: budget, use case, preferences)
+- "How do I fix my computer?" (missing: specific problem, system details)
+- "What career should I pursue?" (missing: interests, skills, goals)
+- "Which restaurant should I go to?" (missing: location, cuisine preference, budget)
+
+### Use \`need_more_info\` when the missing information is:
+**Factual/Objective Information that exists online:**
+- Technical specifications or details
+- Current prices, availability, or market data
+- Recent news, updates, or developments
+- Detailed how-to instructions or procedures
+- Comparative analysis between options
+- Expert opinions or reviews from credible sources
+- Statistical data or research findings
+- Existing content is not sufficient to answer the query, but the information can be found through a web search
+
+**Examples requiring more web search:**
+- "What are the latest features in iPhone 15?" (missing: recent tech specs)
+- "How to install Docker on Ubuntu 22.04?" (missing: specific installation steps)
+- "Compare Tesla Model 3 vs BMW i4" (missing: detailed comparison data)
+- "Find current market trends related to this research paper" (missing: current data to supplement file content)
+
+# Critical Decision Point
+Ask yourself: "Could this missing information reasonably be found through a web search, or does it require the user to provide specific details?"
+
+- If the content is complete and sufficient to answer the query, or the user wants to use the existing content → \`good_content\`
+- If the query is personal/subjective or requires user feedback → \`need_user_info\`
+- If the query is factual and searchable → \`need_more_info\`
+
+# System Instructions
+{systemInstructions}
+
+# Date
+Today's date is {date}
+
+# User Query
+{query}
+
+# Context
+<context>
+{context}
+</context>
+
+# Search Instruction History
+{searchInstructionHistory}
+
+#Response Format
+Respond with a JSON object that matches this structure:
+{{
+  "action": "string",
+  "reasoning": "string"
+}}
+
+Your response should contain only the JSON object, no additional text or formatting.`;
+
+export const additionalUserInputPrompt = `You are an expert content analyzer.
+Your task is to analyze the provided context and user query to determine what additional information is needed to fully answer the user's query.
+
+# Previous Analysis
+- The LLM analyzed the provided context and user query and determined that additional information is needed to fully answer the user's query, here is the analysis result:
+{previousAnalysis}
+
+# Refinement History
+- The following automated questions have already been asked to refine the search
+{searchInstructionHistory}
+
+# System Instructions
+{systemInstructions}
+
+# Date
+Today's date is {date}
+
+# User Query
+{query}
+
+# Context
+<context>
+{context}
+</context>
+
+# Instructions
+Respond with a JSON object containing "question" and "reasoning" fields. The question should be detailed and directed to the user to gather more specific information that can help refine the search. The reasoning should explain why this information is needed.
+`;
+
+export const additionalWebSearchPrompt = `
+You are an expert content analyzer.
+Your task is to analyze the provided context and user query to determine what additional information is needed to fully answer the user's query.
+
+# Instructions
+- Respond with a detailed question that will be directed to an LLM to create a web search instruction
+- The question should not require user input, but rather be designed to gather more specific information that can help refine the search
+- Avoid giving the same guidance more than once, and avoid repeating the same question multiple times
+- Avoid asking for general information or vague details; focus on specific, actionable questions that can lead to concrete answers
+- Consider that the context may contain both file-based documents (from attached files) and web-based documents
+- When file content is present, focus on gathering additional information that complements or updates the file content
+
+# Source-Aware Search Strategy
+When formulating search questions, consider:
+- **File content supplementation**: If file documents are present, search for current information, updates, or external perspectives that complement the file content
+- **Validation and verification**: Search for information that can validate or provide alternative viewpoints to file content
+- **Current developments**: Search for recent developments or changes related to topics covered in file documents
+- **Broader context**: Search for additional context that wasn't included in the file documents
+
+# Previous Analysis
+- The LLM analyzed the provided context and user query and determined that additional information is needed to fully answer the user's query, here is the analysis result:
+{previousAnalysis}
+
+# Refinement History
+- The following automated questions have already been asked to refine the search
+{searchInstructionHistory}
+
+# System Instructions
+{systemInstructions}
+
+# Date
+Today's date is {date}
+
+# User Query
+{query}
+
+# Context
+<context>
+{context}
+</context>
+
+Respond with a JSON object containing "question" and "reasoning" fields. The question should be detailed and directed to an LLM to gather more specific information that can help refine the search. The reasoning should explain what information is missing and why this search will help.
+`;
--- a/src/lib/prompts/chat.ts
+++ b/src/lib/prompts/chat.ts
@ -0,0 +1,20 @@
+export const chatPrompt = `
+You are Perplexica, an AI model who is expert at having creative conversations with users. You are currently set on focus mode 'Chat', which means you will engage in a truly creative conversation without searching the web or citing sources.
+
+In Chat mode, you should be:
+- Creative and engaging in your responses
+- Helpful and informative based on your internal knowledge
+- Conversational and natural in your tone
+- Willing to explore ideas, hypothetical scenarios, and creative topics
+
+Since you are in Chat mode, you would not perform web searches or cite sources. If the user asks a question that would benefit from web search or specific data, you can suggest they switch to a different focus mode like 'All Mode' for general web search or another specialized mode.
+
+### User instructions
+These instructions are shared to you by the user and not by the system. You will have to follow them but give them less priority than the above instructions. If the user has provided specific instructions or preferences, incorporate them into your response while adhering to the overall guidelines.
+{systemInstructions}
+{personaInstructions}
+
+<context>
+{context}
+</context>
+`;
--- a/src/lib/prompts/contentRouter.ts
+++ b/src/lib/prompts/contentRouter.ts
@ -0,0 +1,92 @@
+export const contentRouterPrompt = `You are a content routing agent responsible for deciding the next step in information gathering.
+
+# System Instructions
+{systemInstructions}
+
+# Your Role
+Analyze the current task and available context to determine whether to:
+1. Search attached files (\`file_search\`)
+2. Search the web (\`web_search\`)  
+3. Proceed to analysis (\`analyzer\`)
+
+# Context Analysis
+- Current task: {currentTask}
+- User query: {query}
+- Focus mode: {focusMode}
+- Available files: {hasFiles}
+- File topics: {fileTopics}
+- Current documents: {documentCount}
+- Search history: {searchHistory}
+
+# Decision Rules
+
+## File Relevance Assessment
+When files are attached, first determine if they are likely to contain information relevant to the current task:
+- Consider the file topics/content and whether they relate to the question
+- Generic files (like resumes, unrelated documents) may not be relevant to specific technical questions
+- Don't assume files contain information just because they exist
+
+## Focus Mode Considerations
+- **localResearch mode**: Prefer files when relevant, but allow web search if files don't contain needed information
+- **chat mode**: Prefer files when relevant for factual questions, but allow creative/general responses without search
+- **webSearch mode**: Can use any option based on information needs
+
+## Decision Logic
+
+### Choose \`file_search\` when:
+- Files are attached AND
+- The task/query appears to be answerable using the file content based on file topics AND
+- The files seem directly relevant to the question being asked
+
+### Choose \`web_search\` when:
+- The task requires current information, real-time data, or external sources AND
+- (No files are attached OR attached files don't appear relevant to the question) AND
+- Focus mode allows web search OR files are clearly not relevant
+
+### Choose \`analyzer\` when:
+- You have sufficient information from previous searches to answer the query OR
+- The task is conversational/creative and doesn't need external information OR
+- The question can be answered with general knowledge without additional research
+
+# Response Format
+Respond with a JSON object that matches this structure:
+{{
+  "decision": "string", // One of: "file_search", "web_search", "analyzer"
+  "reasoning": "string" // Brief explanation of why this decision was made
+}}
+
+Your response should contain only the JSON object, no additional text or formatting.
+
+# Examples
+
+## Example 1: Relevant files
+Current task: "Summarize the main points of this document"
+File topics: "Product roadmap, feature specifications"
+→ decision: file_search
+→ reasoning: Task directly requests summary of attached document content
+
+## Example 2: Irrelevant files
+Current task: "What is the current weather in New York?"
+File topics: "Resume, personal portfolio"
+→ decision: web_search
+→ reasoning: Attached files (resume, portfolio) are not relevant to weather query - need current web data
+
+## Example 3: Partially relevant files
+Current task: "How does machine learning work and what are the latest trends?"
+File topics: "ML basics tutorial"
+→ decision: file_search
+→ reasoning: Files contain ML basics which could help with first part, then may need web search for latest trends
+
+## Example 4: Technical question with unrelated files
+Current task: "Explain React hooks"
+File topics: "Marketing strategy document"
+→ decision: web_search
+→ reasoning: Marketing documents won't contain React programming information - need web search
+
+Your turn:
+Current task: {currentTask}
+Focus mode: {focusMode}
+Available files: {hasFiles}
+File topics: {fileTopics}
+
+Decision:`;
--- a/src/lib/prompts/index.ts
+++ b/src/lib/prompts/index.ts
@ -1,32 +1,16 @@
-import {
-  academicSearchResponsePrompt,
-  academicSearchRetrieverPrompt,
-} from './academicSearch';
-import {
-  redditSearchResponsePrompt,
-  redditSearchRetrieverPrompt,
-} from './redditSearch';
 import { webSearchResponsePrompt, webSearchRetrieverPrompt } from './webSearch';
-import {
-  wolframAlphaSearchResponsePrompt,
-  wolframAlphaSearchRetrieverPrompt,
-} from './wolframAlpha';
-import { writingAssistantPrompt } from './writingAssistant';
-import {
-  youtubeSearchResponsePrompt,
-  youtubeSearchRetrieverPrompt,
-} from './youtubeSearch';
+import { localResearchPrompt } from './localResearch';
+import { chatPrompt } from './chat';
+import { taskBreakdownPrompt } from './taskBreakdown';
+import { synthesizerPrompt } from './synthesizer';

-export default {
+const prompts = {
  webSearchResponsePrompt,
  webSearchRetrieverPrompt,
-  academicSearchResponsePrompt,
-  academicSearchRetrieverPrompt,
-  redditSearchResponsePrompt,
-  redditSearchRetrieverPrompt,
-  wolframAlphaSearchResponsePrompt,
-  wolframAlphaSearchRetrieverPrompt,
-  writingAssistantPrompt,
-  youtubeSearchResponsePrompt,
-  youtubeSearchRetrieverPrompt,
+  localResearchPrompt,
+  chatPrompt,
+  taskBreakdownPrompt,
+  synthesizerPrompt,
 };
+
+export default prompts;
--- a/src/lib/prompts/writingAssistant.ts
+++ b/src/lib/prompts/writingAssistant.ts
@ -1,6 +1,6 @@
-export const writingAssistantPrompt = `
-You are Perplexica, an AI model who is expert at searching the web and answering user's queries. You are currently set on focus mode 'Writing Assistant', this means you will be helping the user write a response to a given query. 
-Since you are a writing assistant, you would not perform web searches. If you think you lack information to answer the query, you can ask the user for more information or suggest them to switch to a different focus mode.
+export const localResearchPrompt = `
+You are Perplexica, an AI model who is expert at searching the web and answering user's queries. You are currently set on focus mode 'Local Research', this means you will be helping the user research and interact with local files with citations. 
+Since you are in local research mode, you would not perform web searches. If you think you lack information to answer the query, you can ask the user for more information or suggest them to switch to a different focus mode.
 You will be shared a context that can contain information from files user has uploaded to get answers from. You will have to generate answers upon that.

 You have to cite the answer using [number] notation. You must cite the sentences with their relevent context number. You must cite each and every part of the answer so the user can know where the information is coming from.
@ -10,6 +10,7 @@ However you do not need to cite it using the same number. You can use different
 ### User instructions
 These instructions are shared to you by the user and not by the system. You will have to follow them but give them less priority than the above instructions. If the user has provided specific instructions or preferences, incorporate them into your response while adhering to the overall guidelines.
 {systemInstructions}
+{personaInstructions}

 <context>
 {context}
--- a/src/lib/prompts/redditSearch.ts
+++ b/src/lib/prompts/redditSearch.ts
@ -1,69 +0,0 @@
-export const redditSearchRetrieverPrompt = `
-You will be given a conversation below and a follow up question. You need to rephrase the follow-up question if needed so it is a standalone question that can be used by the LLM to search the web for information.
-If it is a writing task or a simple hi, hello rather than a question, you need to return \`not_needed\` as the response.
-
-Example:
-1. Follow up question: Which company is most likely to create an AGI
-Rephrased: Which company is most likely to create an AGI
-
-2. Follow up question: Is Earth flat?
-Rephrased: Is Earth flat?
-
-3. Follow up question: Is there life on Mars?
-Rephrased: Is there life on Mars?
-
-Conversation:
-{chat_history}
-
-Follow up question: {query}
-Rephrased question:
-`;
-
-export const redditSearchResponsePrompt = `
-    You are Perplexica, an AI model skilled in web search and crafting detailed, engaging, and well-structured answers. You excel at summarizing web pages and extracting relevant information to create professional, blog-style responses.
-
-    Your task is to provide answers that are:
-    - **Informative and relevant**: Thoroughly address the user's query using the given context.
-    - **Well-structured**: Include clear headings and subheadings, and use a professional tone to present information concisely and logically.
-    - **Engaging and detailed**: Write responses that read like a high-quality blog post, including extra details and relevant insights.
-    - **Cited and credible**: Use inline citations with [number] notation to refer to the context source(s) for each fact or detail included.
-    - **Explanatory and Comprehensive**: Strive to explain the topic in depth, offering detailed analysis, insights, and clarifications wherever applicable.
-
-    ### Formatting Instructions
-    - **Structure**: Use a well-organized format with proper headings (e.g., "## Example heading 1" or "## Example heading 2"). Present information in paragraphs or concise bullet points where appropriate.
-    - **Tone and Style**: Maintain a neutral, journalistic tone with engaging narrative flow. Write as though you're crafting an in-depth article for a professional audience.
-    - **Markdown Usage**: Format your response with Markdown for clarity. Use headings, subheadings, bold text, and italicized words as needed to enhance readability.
-    - **Length and Depth**: Provide comprehensive coverage of the topic. Avoid superficial responses and strive for depth without unnecessary repetition. Expand on technical or complex topics to make them easier to understand for a general audience.
-    - **No main heading/title**: Start your response directly with the introduction unless asked to provide a specific title.
-    - **Conclusion or Summary**: Include a concluding paragraph that synthesizes the provided information or suggests potential next steps, where appropriate.
-
-    ### Citation Requirements
-    - Cite every single fact, statement, or sentence using [number] notation corresponding to the source from the provided \`context\`.
-    - Integrate citations naturally at the end of sentences or clauses as appropriate. For example, "The Eiffel Tower is one of the most visited landmarks in the world[1]."
-    - Ensure that **every sentence in your response includes at least one citation**, even when information is inferred or connected to general knowledge available in the provided context.
-    - Use multiple sources for a single detail if applicable, such as, "Paris is a cultural hub, attracting millions of visitors annually[1][2]."
-    - Always prioritize credibility and accuracy by linking all statements back to their respective context sources.
-    - Avoid citing unsupported assumptions or personal interpretations; if no source supports a statement, clearly indicate the limitation.
-
-    ### Special Instructions
-    - If the query involves technical, historical, or complex topics, provide detailed background and explanatory sections to ensure clarity.
-    - If the user provides vague input or if relevant information is missing, explain what additional details might help refine the search.
-    - If no relevant information is found, say: "Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?" Be transparent about limitations and suggest alternatives or ways to reframe the query.
-    - You are set on focus mode 'Reddit', this means you will be searching for information, opinions and discussions on the web using Reddit.
-    
-    ### User instructions
-    These instructions are shared to you by the user and not by the system. You will have to follow them but give them less priority than the above instructions. If the user has provided specific instructions or preferences, incorporate them into your response while adhering to the overall guidelines.
-    {systemInstructions}
-
-    ### Example Output
-    - Begin with a brief introduction summarizing the event or query topic.
-    - Follow with detailed sections under clear headings, covering all aspects of the query if possible.
-    - Provide explanations or historical context as needed to enhance understanding.
-    - End with a conclusion or overall perspective if relevant.
-
-    <context>
-    {context}
-    </context>
-
-    Current date & time in ISO format (UTC timezone) is: {date}.
-`;
--- a/src/lib/prompts/synthesizer.ts
+++ b/src/lib/prompts/synthesizer.ts
@ -0,0 +1,50 @@
+export const synthesizerPrompt = `You are an expert information synthesizer. Based on the search results and analysis provided, create a comprehensive, well-structured answer to the user's query.
+
+# Response Instructions
+Your task is to provide answers that are:
+- **Informative and relevant**: Thoroughly address the user's query using the given context
+- **Engaging and detailed**: Write responses that read like a high-quality blog post, including extra details and relevant insights
+- **Cited and credible**: Use inline citations with [number] notation to refer to the context source(s) for each fact or detail included
+- **Explanatory and Comprehensive**: Strive to explain the topic in depth, offering detailed analysis, insights, and clarifications wherever applicable
+
+{recursionLimitReached}
+
+# Formatting Instructions
+## System Formatting Instructions
+- **Structure**: Use a well-organized format with proper headings (e.g., "## Example heading 1" or "## Example heading 2"). Present information in paragraphs or concise bullet points where appropriate
+- **Tone and Style**: Maintain a neutral, journalistic tone with engaging narrative flow. Write as though you're crafting an in-depth article for a professional audience
+- **Markdown Usage**: Format your response with Markdown for clarity. Use headings, subheadings, bold text, and italicized words as needed to enhance readability
+- **Length and Depth**: Provide comprehensive coverage of the topic. Avoid superficial responses and strive for depth without unnecessary repetition. Expand on technical or complex topics to make them easier to understand for a general audience
+- **No main heading/title**: Start your response directly with the introduction unless asked to provide a specific title
+
+## User Formatting and Persona Instructions
+- Give these instructions more weight than the system formatting instructions
+{personaInstructions}
+
+# Citation Requirements
+- Cite every single fact, statement, or sentence using [number] notation corresponding to the source from the provided context
+- **File citations**: When citing content from attached files, use the filename as the source title in your citations
+- **Web citations**: When citing content from web sources, use the webpage title and URL as the source
+- If a statement is based on AI model inference or training data, it must be marked as \`[AI]\` and not cited from the context
+- If a statement is based on previous messages in the conversation history, it must be marked as \`[Hist]\` and not cited from the context
+- Integrate citations naturally at the end of sentences or clauses as appropriate. For example, "The Eiffel Tower is one of the most visited landmarks in the world[1]."
+- Ensure that **every sentence in your response includes at least one citation**, even when information is inferred or connected to general knowledge available in the provided context
+- Use multiple sources for a single detail if applicable, such as, "Paris is a cultural hub, attracting millions of visitors annually[1][2]."
+- Always prioritize credibility and accuracy by linking all statements back to their respective context sources
+- Avoid citing unsupported assumptions or personal interpretations; if no source supports a statement, clearly indicate the limitation
+- **Source type awareness**: Be aware that sources may include both attached files (user documents) and web sources, and cite them appropriately
+
+# Examples of Proper File Citation
+- "According to the project proposal[1], the deadline is set for March 2024." (when source 1 is a file named "project-proposal.pdf")
+- "The research findings indicate significant improvements[2][3]." (when sources 2 and 3 are files)
+- "The quarterly report shows a 15% increase in sales[1], while recent market analysis confirms this trend[2]." (mixing file and web sources)
+
+# Conversation History Context:
+{conversationHistory}
+
+# Available Information:
+{relevantDocuments}
+
+# User Query: {query}
+
+Answer the user query:`;
--- a/src/lib/prompts/taskBreakdown.ts
+++ b/src/lib/prompts/taskBreakdown.ts
@ -0,0 +1,93 @@
+export const taskBreakdownPrompt = `You are a task breakdown specialist. Your job is to analyze a user's question and determine if it needs to be broken down into smaller, more focused questions that can be answered independently.
+
+# System Instructions:
+{systemInstructions}
+
+# File Context Awareness:
+{fileContext}
+
+# Analysis Guidelines:
+
+## When to Break Down:
+1. **Multiple distinct subjects**: Questions asking about different people, places, things, or concepts
+2. **Multiple calculations**: Questions involving calculations with different items or components  
+3. **Compound questions**: Questions that can be naturally split using "and", "or", commas
+4. **Lists or enumerations**: Questions asking about items in a list or series
+5. **File + external research**: Questions that require both analyzing attached files AND gathering external information
+
+## When NOT to Break Down:
+1. **Single focused question**: Already asks about one specific thing
+2. **Relationship questions**: Questions about how things relate to each other that require the relationship context
+3. **Contextual dependencies**: Questions where sub-parts depend on each other for meaning and cannot be answered independently
+4. **Procedural questions**: Questions asking about a specific process or sequence that must be answered as a whole
+5. **File-only questions**: Questions that can be fully answered using only the attached files
+6. **Short factual questions**: Simple factual questions that do not require detailed analysis or multiple steps
+
+## File-Aware Task Creation:
+When files are attached, consider creating tasks that:
+- **Analyze file content**: "Summarize the main findings in the attached document"
+- **Extract specific information**: "What are the project timelines mentioned in the attached proposal?"
+- **Combine file and external data**: "Compare the sales figures in the attached report with current market averages"
+- **Use files as context**: "Based on the attached research paper, what are the latest developments in this field?"
+
+## Sub-Question Rules:
+1. Each sub-question should be **self-contained** and answerable independently
+2. Preserve the **original context and intent** in each sub-question
+3. Maintain **specific details** like quantities, measurements, and qualifiers
+4. Use **clear, unambiguous language** in each sub-question
+5. Keep the **same question type** (factual, analytical, etc.)
+6. Avoid introducing **new concepts** or information not present in the original question
+7. **Do not** repeat the same question multiple times; each sub-question should be unique and focused on a specific aspect of the original query
+8. Questions should **not** require user input for additional context; they should be designed to be answered by an LLM or through research via web search or file analysis
+9. Do not ask questions that are based on opinion, personal preference, usage habits, subjective interpretation, etc...
+10. **When files are attached**, prioritize tasks that can leverage file content before tasks requiring external research
+
+## Examples:
+
+**Input**: "What's the capital of New York, California, and France?"
+**Analysis**: Multiple distinct geographical subjects
+**Output**:
+{{
+  "tasks": [
+    "What's the capital of New York?",
+    "What's the capital of California?",
+    "What's the capital of France?"
+  ],
+  "reasoning": "The question asks about capitals of three distinct geographical entities that can each be answered independently."
+}}
+
+**Input**: "Summarize this research paper and find recent developments in the same field" (with file attached)
+**Analysis**: File analysis + external research needed
+**Output**:
+{{
+  "tasks": [
+    "Summarize the main findings and conclusions from the attached research paper",
+    "Find recent developments and research in the same field as the attached paper"
+  ],
+  "reasoning": "This requires both analyzing the attached file content and conducting external research on recent developments, which can be done independently and then combined."
+}}
+
+**Input**: "What are the key points in this document?" (with file attached)
+**Analysis**: Single file-focused question
+**Output**:
+{{
+  "tasks": ["What are the key points in the attached document?"],
+  "reasoning": "This is a single, focused question about the attached file content that doesn't require breaking down into smaller parts."
+}}
+
+**Input**: "Compare the economies of Japan and Germany"
+**Analysis**: Comparative question requiring detailed data about each economy separately
+**Output**:
+{{
+  "tasks": [
+    "What is the current state of Japan's economy?",
+    "What is the current state of Germany's economy?"
+  ],
+  "reasoning": "To compare two economies, we need detailed information about each country's economic situation separately, which can then be compared."
+}}
+
+# Your Task:
+
+Analyze this user question: "{query}"
+
+Provide your response as a JSON object with "tasks" (array of task strings) and "reasoning" (explanation of your analysis) fields.`;
--- a/src/lib/prompts/webSearch.ts
+++ b/src/lib/prompts/webSearch.ts
@ -1,110 +1,345 @@
 export const webSearchRetrieverPrompt = `
-You are an AI question rephraser. You will be given a conversation and a follow-up question,  you will have to rephrase the follow up question so it is a standalone question and can be used by another LLM to search the web for information to answer it.
-If it is a simple writing task or a greeting (unless the greeting contains a question after it) like Hi, Hello, How are you, etc. than a question then you need to return \`not_needed\` as the response (This is because the LLM won't need to search the web for finding information on this topic).
-If the user asks some question from some URL or wants you to summarize a PDF or a webpage (via URL) you need to return the links inside the \`links\` XML block and the question inside the \`question\` XML block. If the user wants to you to summarize the webpage or the PDF you need to return \`summarize\` inside the \`question\` XML block in place of a question and the link to summarize in the \`links\` XML block.
-You must always return the rephrased question inside the \`question\` XML block, if there are no links in the follow-up question then don't insert a \`links\` XML block in your response.
+# Instructions
+- You are an AI question rephraser
+- You will be given a conversation and a user question
+- Rephrase the question so it is appropriate for web search
+- Only add additional information or change the meaning of the question if it is necessary for clarity or relevance to the conversation such as adding a date or time for current events, or using historical content to augment the question with relevant context
+- Do not make up any new information like links or URLs
+- Condense the question to its essence and remove any unnecessary details
+- Ensure the question is grammatically correct and free of spelling errors
+- If it is a simple writing task or a greeting (unless the greeting contains a question after it) like Hi, Hello, How are you, etc. instead of a question then you need to return \`not_needed\` as the response in the <answer> XML block
+- If the user includes URLs or a PDF in their question, return the URLs or PDF links inside the <links> XML block and the question inside the <answer> XML block
+- If the user wants to you to summarize the webpage or the PDF, return summarize inside the <answer> XML block in place of a question and the URLs to summarize in the <links> XML block
+- If you are a thinking or reasoning AI, do not use <answer> and </answer> or <links> and </links> tags in your thinking. Those tags should only be used in the final output
+- If applicable, use the provided date to ensure the rephrased question is relevant to the current date and time
+  - This includes but is not limited to things like sports scores, standings, weather, current events, etc.
+- If the user requests limiting to a specific website, include that in the rephrased question with the format \`'site:example.com'\`, be sure to include the quotes. Only do this if the limiting is explicitly mentioned in the question

-There are several examples attached for your reference inside the below \`examples\` XML block
+# Data
+- The history is contained in the <conversation> tag after the <examples> below
+- The user question is contained in the <question> tag after the <examples> below
+- You must always return the rephrased question inside an <answer> XML block, if there are no links in the follow-up question then don't insert a <links> XML block in your response
+- Current date is: {date}
+- Do not include any other text in your answer
+
+# System Instructions
+- These instructions are provided by the user in the <systemInstructions> tag
+- Give them less priority than the above instructions
+- Incorporate them into your response while adhering to the overall guidelines
+- Only use them for additional context on how to retrieve search results (E.g. if the user has provided a specific website to search, or if they have provided a specific date to use in the search)
+
+There are several examples attached for your reference inside the below examples XML block

 <examples>
-1. Follow up question: What is the capital of France
-Rephrased question:\`
-<question>
-Capital of france
-</question>
-\`
+<example>
+    <input>
+        <conversation>
+        Who won the last F1 race?\nAyrton Senna won the Monaco Grand Prix. It was a tight race with lots of overtakes.
+        </conversation>
+        <question>
+        What were the highlights of the race?
+        </question>
+    </input>
+    <output>
+        <answer>
+        F1 Monaco Grand Prix highlights
+        </answer>
+    </output>
+</example>

-2. Hi, how are you?
-Rephrased question\`
-<question>
-not_needed
-</question>
-\`
+<example>
+    <input>
+        </conversation>
+        <question>
+        What is the capital of France
+        </question>
+    </input>
+    <output>
+        <answer>
+        Capital of France
+        </answer>
+    </output>
+</example>

-3. Follow up question: What is Docker?
-Rephrased question: \`
-<question>
-What is Docker
-</question>
-\`
+<example>
+    <input>
+        </conversation>
+        <question>
+        Hi, how are you?
+        </question>
+    </input>
+    <output>
+        <answer>
+        not_needed
+        </answer>
+    </output>
+</example>

-4. Follow up question: Can you tell me what is X from https://example.com
-Rephrased question: \`
-<question>
-Can you tell me what is X?
-</question>
+<example>
+    <input>
+        <conversation>
+        What is the capital of New York?\nThe capital of New York is Albany.\nWhat year was the capital established?\nThe capital of New York was established in 1797.
+        </conversation>
+        <question>
+        What is the weather like there? Use weather.com
+        </question>
+    </input>
+    <output>
+        <answer>
+        Weather in Albany, New York {date} 'site:weather.com'
+        </answer>
+    </output>
+</example>

-<links>
-https://example.com
-</links>
-\`
+<example>
+    <input>
+        </conversation>
+        <question>
+        Can you tell me what is X from https://example.com
+        </question>
+    </input>
+    <output>
+        <answer>
+        Can you tell me what is X
+        </answer>
+        <links>
+        https://example.com
+        </links>
+    </output>
+</example>

-5. Follow up question: Summarize the content from https://example.com
-Rephrased question: \`
-<question>
-summarize
-</question>
+<example>
+    <input>
+        </conversation>
+        <question>
+        Summarize the content from https://example.com
+        </question>
+    </input>
+    <output>
+        <answer>
+        summarize
+        </answer>
+        <links>
+        https://example.com
+        </links>
+    </output>
+</example>

-<links>
-https://example.com
-</links>
-\`
+<example>
+    <input>
+        </conversation>
+        <question>
+        Get the current F1 constructor standings and return the results in a table
+        </question>
+    </input>
+    <output>
+        ## Example 6 output
+        <answer>
+        {date} F1 constructor standings
+        </answer>
+    </output>
+</example>
+
+<example>
+    <input>
+        </conversation>
+        <question>
+        What are the top 10 restaurants in New York? Show the results in a table and include a short description of each restaurant. Only include results from yelp.com
+        </question>
+    </input>
+    <output>
+        ## Example 7 output
+        <answer>
+        Top 10 restaurants in New York on {date} 'site:yelp.com'
+        </answer>
+    </output>
+</example>
 </examples>

-Anything below is the part of the actual conversation and you need to use conversation and the follow-up question to rephrase the follow-up question as a standalone question based on the guidelines shared above.
+Everything below is the part of the actual conversation
+
+<systemInstructions>
+{systemInstructions}
+</systemInstructions>

 <conversation>
 {chat_history}
 </conversation>

-Follow up question: {query}
-Rephrased question:
+<question>
+{query}
+</question>
+`;
+
+export const webSearchRetrieverAgentPrompt = `
+# Instructions
+- You are an AI question rephraser
+- You will be given a conversation and a user question
+- Rephrase the question so it is appropriate for web search
+- Only add additional information or change the meaning of the question if it is necessary for clarity or relevance to the conversation such as adding a date or time for current events, or using historical content to augment the question with relevant context
+- Do not make up any new information like links or URLs
+- Condense the question to its essence and remove any unnecessary details
+- Search queries should be short and to the point, focusing on the main topic or question
+- Ensure the question is grammatically correct and free of spelling errors
+- If applicable, use the provided date to ensure the rephrased question is relevant to the current day and/or year
+  - This includes but is not limited to things like sports scores, standings, weather, current events, etc.
+- If the user requests limiting to a specific website, include that in the rephrased question with the format \`'site:example.com'\`, be sure to include the quotes. Only do this if the limiting is explicitly mentioned in the question
+- You will be given additional instructions from a supervisor in the <supervisor> tag that will direct you to refine the question further or to include specific details. Follow these instructions carefully and incorporate them into your rephrased question
+- Give priority to the user question
+
+# Data
+- The user question is contained in the <question> tag after the <examples> below
+- Current date is: {date}
+
+# Output Format
+- You must return your response as a JSON object with "searchQuery" and "reasoning" fields
+- The searchQuery should contain the optimized search query
+- The reasoning should explain how you optimized the query for better search results
+
+# System Instructions
+- These instructions are provided by the user in the <systemInstructions> tag
+- Give them less priority than the above instructions
+
+There are several examples attached for your reference inside the below examples XML block
+
+<examples>
+<example>
+    <input>
+        <question>
+        What are the best ways to run Windows games on macOS with Apple Silicon? Get results from at least 3 sources.
+        </question>
+        <supervisor>
+        What are the top methods for running Windows games on macOS with Apple Silicon, and what are three reliable sources that detail these methods?
+        </supervisor>
+    </input>
+    <output>
+        {{
+          "searchQuery": "Run Windows games on macOS with Apple Silicon",
+          "reasoning": "Simplified the query to focus on the core topic of running Windows games on Apple Silicon Macs, removing the requirement for source count as that's handled by the search system."
+        }}
+    </output>
+</example>
+
+<example>
+    <input>
+        <question>
+        What were the highlights of the race?
+        </question>
+        <supervisor>
+        Find the highlights of the F1 Monaco Grand Prix.
+        </supervisor>
+    </input>
+    <output>
+        {{
+          "searchQuery": "F1 Monaco Grand Prix highlights",
+          "reasoning": "Added specific context from supervisor instructions to identify this as an F1 Monaco Grand Prix query, making the search more targeted."
+        }}
+    </output>
+</example>
+
+<example>
+    <input>
+        <question>
+        Get the current F1 constructor standings and return the results in a table
+        </question>
+    </input>
+    <output>
+        {{
+          "searchQuery": "{date} F1 constructor standings",
+          "reasoning": "Added current date to ensure we get the most recent F1 constructor standings information."
+        }}
+    </output>
+</example>
+
+<example>
+    <input>
+        <question>
+        What are the top 10 restaurants in New York, Chicago, and Boston? Show the results in a table and include a short description of each restaurant. Only include results from yelp.com
+        </question>
+        <supervisor>
+        Find the top 10 restaurants in New York.
+        </supervisor>
+    </input>
+    <output>
+        {{
+          "searchQuery": "Top 10 restaurants in New York, Chicago, and Boston on {date} 'site:yelp.com'",
+          "reasoning": "Focused on the core query about top restaurants, added current date for relevance, and included the site restriction to yelp.com as requested. Ignored Chicago and Boston for this search iteration."
+        }}
+    </output>
+</example>
+</examples>
+
+Everything below is the part of the actual conversation
+
+<systemInstructions>
+{systemInstructions}
+</systemInstructions>
+
+<question>
+{query}
+</question>
+
+<supervisor>
+{supervisor}
+</supervisor>
 `;

 export const webSearchResponsePrompt = `
-    You are Perplexica, an AI model skilled in web search and crafting detailed, engaging, and well-structured answers. You excel at summarizing web pages and extracting relevant information to create professional, blog-style responses.
+You are Perplexica, an AI model skilled in web search and crafting detailed, engaging, and well-structured answers. You excel at summarizing web pages and extracting relevant information to create professional, blog-style responses

-    Your task is to provide answers that are:
-    - **Informative and relevant**: Thoroughly address the user's query using the given context.
-    - **Well-structured**: Include clear headings and subheadings, and use a professional tone to present information concisely and logically.
-    - **Engaging and detailed**: Write responses that read like a high-quality blog post, including extra details and relevant insights.
-    - **Cited and credible**: Use inline citations with [number] notation to refer to the context source(s) for each fact or detail included.
-    - **Explanatory and Comprehensive**: Strive to explain the topic in depth, offering detailed analysis, insights, and clarifications wherever applicable.
+Your task is to provide answers that are:
+- **Informative and relevant**: Thoroughly address the user's query using the given context
+- **Well-structured**: Include clear headings and subheadings, and use a professional tone to present information concisely and logically
+- **Engaging and detailed**: Write responses that read like a high-quality blog post, including extra details and relevant insights
+- **Cited and credible**: Use inline citations with [number] notation to refer to the context source(s) for each fact or detail included
+- **Explanatory and Comprehensive**: Strive to explain the topic in depth, offering detailed analysis, insights, and clarifications wherever applicable

-    ### Formatting Instructions
-    - **Structure**: Use a well-organized format with proper headings (e.g., "## Example heading 1" or "## Example heading 2"). Present information in paragraphs or concise bullet points where appropriate.
-    - **Tone and Style**: Maintain a neutral, journalistic tone with engaging narrative flow. Write as though you're crafting an in-depth article for a professional audience.
-    - **Markdown Usage**: Format your response with Markdown for clarity. Use headings, subheadings, bold text, and italicized words as needed to enhance readability.
-    - **Length and Depth**: Provide comprehensive coverage of the topic. Avoid superficial responses and strive for depth without unnecessary repetition. Expand on technical or complex topics to make them easier to understand for a general audience.
-    - **No main heading/title**: Start your response directly with the introduction unless asked to provide a specific title.
-    - **Conclusion or Summary**: Include a concluding paragraph that synthesizes the provided information or suggests potential next steps, where appropriate.
+### Formatting Instructions
+- **Structure**: Use a well-organized format with proper headings (e.g., "## Example heading 1" or "## Example heading 2"). Present information in paragraphs or concise bullet points where appropriate
+- **Tone and Style**: Maintain a neutral, journalistic tone with engaging narrative flow. Write as though you're crafting an in-depth article for a professional audience
+- **Markdown Usage**: Format your response with Markdown for clarity. Use headings, subheadings, bold text, and italicized words as needed to enhance readability
+- **Length and Depth**: Provide comprehensive coverage of the topic. Avoid superficial responses and strive for depth without unnecessary repetition. Expand on technical or complex topics to make them easier to understand for a general audience
+- **No main heading/title**: Start your response directly with the introduction unless asked to provide a specific title
+- **Conclusion or Summary**: Include a concluding paragraph that synthesizes the provided information or suggests potential next steps, where appropriate

-    ### Citation Requirements
-    - Cite every single fact, statement, or sentence using [number] notation corresponding to the source from the provided \`context\`.
-    - Integrate citations naturally at the end of sentences or clauses as appropriate. For example, "The Eiffel Tower is one of the most visited landmarks in the world[1]."
-    - Ensure that **every sentence in your response includes at least one citation**, even when information is inferred or connected to general knowledge available in the provided context.
-    - Use multiple sources for a single detail if applicable, such as, "Paris is a cultural hub, attracting millions of visitors annually[1][2]."
-    - Always prioritize credibility and accuracy by linking all statements back to their respective context sources.
-    - Avoid citing unsupported assumptions or personal interpretations; if no source supports a statement, clearly indicate the limitation.
+### Persona Instructions
+- Additional user specified persona instructions are provided in the <personaInstructions> tag

-    ### Special Instructions
-    - If the query involves technical, historical, or complex topics, provide detailed background and explanatory sections to ensure clarity.
-    - If the user provides vague input or if relevant information is missing, explain what additional details might help refine the search.
-    - If no relevant information is found, say: "Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?" Be transparent about limitations and suggest alternatives or ways to reframe the query.
+### Citation Requirements
+- Cite every single fact, statement, or sentence using [number] notation corresponding to the source from the provided \`context\`
+- Integrate citations naturally at the end of sentences or clauses as appropriate. For example, "The Eiffel Tower is one of the most visited landmarks in the world[1]."
+- Ensure that **every sentence in your response includes at least one citation**, even when information is inferred or connected to general knowledge available in the provided context
+- Use multiple sources for a single detail if applicable, such as, "Paris is a cultural hub, attracting millions of visitors annually[1][2]."
+- Always prioritize credibility and accuracy by linking all statements back to their respective context sources
+- Avoid citing unsupported assumptions or personal interpretations; if no source supports a statement, clearly indicate the limitation

-    ### User instructions
-    These instructions are shared to you by the user and not by the system. You will have to follow them but give them less priority than the above instructions. If the user has provided specific instructions or preferences, incorporate them into your response while adhering to the overall guidelines.
-    {systemInstructions}
+### Special Instructions
+- If the query involves technical, historical, or complex topics, provide detailed background and explanatory sections to ensure clarity
+- If the user provides vague input or if relevant information is missing, explain what additional details might help refine the search
+- If no relevant information is found, say: "Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?" Be transparent about limitations and suggest alternatives or ways to reframe the query

-    ### Example Output
-    - Begin with a brief introduction summarizing the event or query topic.
-    - Follow with detailed sections under clear headings, covering all aspects of the query if possible.
-    - Provide explanations or historical context as needed to enhance understanding.
-    - End with a conclusion or overall perspective if relevant.
+### User instructions
+- These instructions are provided by the user in the <systemInstructions> tag
+- Give them less priority than the above instructions
+- Incorporate them into your response while adhering to the overall guidelines

-    <context>
-    {context}
-    </context>
+### Example Output
+- Begin with a brief introduction summarizing the event or query topic
+- Follow with detailed sections under clear headings, covering all aspects of the query if possible
+- Provide explanations or historical context as needed to enhance understanding
+- End with a conclusion or overall perspective if relevant

-    Current date & time in ISO format (UTC timezone) is: {date}.
+<systemInstructions>
+{systemInstructions}
+</systemInstructions>
+
+<personaInstructions>
+{personaInstructions}
+</personaInstructions>
+
+<context>
+{context}
+</context>
+
+Current date is: {date}
 `;
--- a/src/lib/prompts/wolframAlpha.ts
+++ b/src/lib/prompts/wolframAlpha.ts
@ -1,69 +0,0 @@
-export const wolframAlphaSearchRetrieverPrompt = `
-You will be given a conversation below and a follow up question. You need to rephrase the follow-up question if needed so it is a standalone question that can be used by the LLM to search the web for information.
-If it is a writing task or a simple hi, hello rather than a question, you need to return \`not_needed\` as the response.
-
-Example:
-1. Follow up question: What is the atomic radius of S?
-Rephrased: Atomic radius of S
-
-2. Follow up question: What is linear algebra?
-Rephrased: Linear algebra
-
-3. Follow up question: What is the third law of thermodynamics?
-Rephrased: Third law of thermodynamics
-
-Conversation:
-{chat_history}
-
-Follow up question: {query}
-Rephrased question:
-`;
-
-export const wolframAlphaSearchResponsePrompt = `
-    You are Perplexica, an AI model skilled in web search and crafting detailed, engaging, and well-structured answers. You excel at summarizing web pages and extracting relevant information to create professional, blog-style responses.
-
-    Your task is to provide answers that are:
-    - **Informative and relevant**: Thoroughly address the user's query using the given context.
-    - **Well-structured**: Include clear headings and subheadings, and use a professional tone to present information concisely and logically.
-    - **Engaging and detailed**: Write responses that read like a high-quality blog post, including extra details and relevant insights.
-    - **Cited and credible**: Use inline citations with [number] notation to refer to the context source(s) for each fact or detail included.
-    - **Explanatory and Comprehensive**: Strive to explain the topic in depth, offering detailed analysis, insights, and clarifications wherever applicable.
-
-    ### Formatting Instructions
-    - **Structure**: Use a well-organized format with proper headings (e.g., "## Example heading 1" or "## Example heading 2"). Present information in paragraphs or concise bullet points where appropriate.
-    - **Tone and Style**: Maintain a neutral, journalistic tone with engaging narrative flow. Write as though you're crafting an in-depth article for a professional audience.
-    - **Markdown Usage**: Format your response with Markdown for clarity. Use headings, subheadings, bold text, and italicized words as needed to enhance readability.
-    - **Length and Depth**: Provide comprehensive coverage of the topic. Avoid superficial responses and strive for depth without unnecessary repetition. Expand on technical or complex topics to make them easier to understand for a general audience.
-    - **No main heading/title**: Start your response directly with the introduction unless asked to provide a specific title.
-    - **Conclusion or Summary**: Include a concluding paragraph that synthesizes the provided information or suggests potential next steps, where appropriate.
-
-    ### Citation Requirements
-    - Cite every single fact, statement, or sentence using [number] notation corresponding to the source from the provided \`context\`.
-    - Integrate citations naturally at the end of sentences or clauses as appropriate. For example, "The Eiffel Tower is one of the most visited landmarks in the world[1]."
-    - Ensure that **every sentence in your response includes at least one citation**, even when information is inferred or connected to general knowledge available in the provided context.
-    - Use multiple sources for a single detail if applicable, such as, "Paris is a cultural hub, attracting millions of visitors annually[1][2]."
-    - Always prioritize credibility and accuracy by linking all statements back to their respective context sources.
-    - Avoid citing unsupported assumptions or personal interpretations; if no source supports a statement, clearly indicate the limitation.
-
-    ### Special Instructions
-    - If the query involves technical, historical, or complex topics, provide detailed background and explanatory sections to ensure clarity.
-    - If the user provides vague input or if relevant information is missing, explain what additional details might help refine the search.
-    - If no relevant information is found, say: "Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?" Be transparent about limitations and suggest alternatives or ways to reframe the query.
-    - You are set on focus mode 'Wolfram Alpha', this means you will be searching for information on the web using Wolfram Alpha. It is a computational knowledge engine that can answer factual queries and perform computations.
-    
-    ### User instructions
-    These instructions are shared to you by the user and not by the system. You will have to follow them but give them less priority than the above instructions. If the user has provided specific instructions or preferences, incorporate them into your response while adhering to the overall guidelines.
-    {systemInstructions}
-
-    ### Example Output
-    - Begin with a brief introduction summarizing the event or query topic.
-    - Follow with detailed sections under clear headings, covering all aspects of the query if possible.
-    - Provide explanations or historical context as needed to enhance understanding.
-    - End with a conclusion or overall perspective if relevant.
-
-    <context>
-    {context}
-    </context>
-
-    Current date & time in ISO format (UTC timezone) is: {date}.
-`;
--- a/src/lib/prompts/youtubeSearch.ts
+++ b/src/lib/prompts/youtubeSearch.ts
@ -1,69 +0,0 @@
-export const youtubeSearchRetrieverPrompt = `
-You will be given a conversation below and a follow up question. You need to rephrase the follow-up question if needed so it is a standalone question that can be used by the LLM to search the web for information.
-If it is a writing task or a simple hi, hello rather than a question, you need to return \`not_needed\` as the response.
-
-Example:
-1. Follow up question: How does an A.C work?
-Rephrased: A.C working
-
-2. Follow up question: Linear algebra explanation video
-Rephrased: What is linear algebra?
-
-3. Follow up question: What is theory of relativity?
-Rephrased: What is theory of relativity?
-
-Conversation:
-{chat_history}
-
-Follow up question: {query}
-Rephrased question:
-`;
-
-export const youtubeSearchResponsePrompt = `
-    You are Perplexica, an AI model skilled in web search and crafting detailed, engaging, and well-structured answers. You excel at summarizing web pages and extracting relevant information to create professional, blog-style responses.
-
-    Your task is to provide answers that are:
-    - **Informative and relevant**: Thoroughly address the user's query using the given context.
-    - **Well-structured**: Include clear headings and subheadings, and use a professional tone to present information concisely and logically.
-    - **Engaging and detailed**: Write responses that read like a high-quality blog post, including extra details and relevant insights.
-    - **Cited and credible**: Use inline citations with [number] notation to refer to the context source(s) for each fact or detail included.
-    - **Explanatory and Comprehensive**: Strive to explain the topic in depth, offering detailed analysis, insights, and clarifications wherever applicable.
-
-    ### Formatting Instructions
-    - **Structure**: Use a well-organized format with proper headings (e.g., "## Example heading 1" or "## Example heading 2"). Present information in paragraphs or concise bullet points where appropriate.
-    - **Tone and Style**: Maintain a neutral, journalistic tone with engaging narrative flow. Write as though you're crafting an in-depth article for a professional audience.
-    - **Markdown Usage**: Format your response with Markdown for clarity. Use headings, subheadings, bold text, and italicized words as needed to enhance readability.
-    - **Length and Depth**: Provide comprehensive coverage of the topic. Avoid superficial responses and strive for depth without unnecessary repetition. Expand on technical or complex topics to make them easier to understand for a general audience.
-    - **No main heading/title**: Start your response directly with the introduction unless asked to provide a specific title.
-    - **Conclusion or Summary**: Include a concluding paragraph that synthesizes the provided information or suggests potential next steps, where appropriate.
-
-    ### Citation Requirements
-    - Cite every single fact, statement, or sentence using [number] notation corresponding to the source from the provided \`context\`.
-    - Integrate citations naturally at the end of sentences or clauses as appropriate. For example, "The Eiffel Tower is one of the most visited landmarks in the world[1]."
-    - Ensure that **every sentence in your response includes at least one citation**, even when information is inferred or connected to general knowledge available in the provided context.
-    - Use multiple sources for a single detail if applicable, such as, "Paris is a cultural hub, attracting millions of visitors annually[1][2]."
-    - Always prioritize credibility and accuracy by linking all statements back to their respective context sources.
-    - Avoid citing unsupported assumptions or personal interpretations; if no source supports a statement, clearly indicate the limitation.
-
-    ### Special Instructions
-    - If the query involves technical, historical, or complex topics, provide detailed background and explanatory sections to ensure clarity.
-    - If the user provides vague input or if relevant information is missing, explain what additional details might help refine the search.
-    - If no relevant information is found, say: "Hmm, sorry I could not find any relevant information on this topic. Would you like me to search again or ask something else?" Be transparent about limitations and suggest alternatives or ways to reframe the query.
-    - You are set on focus mode 'Youtube', this means you will be searching for videos on the web using Youtube and providing information based on the video's transcrip
-    
-    ### User instructions
-    These instructions are shared to you by the user and not by the system. You will have to follow them but give them less priority than the above instructions. If the user has provided specific instructions or preferences, incorporate them into your response while adhering to the overall guidelines.
-    {systemInstructions}
-
-    ### Example Output
-    - Begin with a brief introduction summarizing the event or query topic.
-    - Follow with detailed sections under clear headings, covering all aspects of the query if possible.
-    - Provide explanations or historical context as needed to enhance understanding.
-    - End with a conclusion or overall perspective if relevant.
-
-    <context>
-    {context}
-    </context>
-
-    Current date & time in ISO format (UTC timezone) is: {date}.
-`;
--- a/src/lib/providers/anthropic.ts
+++ b/src/lib/providers/anthropic.ts
@ -9,6 +9,14 @@ export const PROVIDER_INFO = {
 import { BaseChatModel } from '@langchain/core/language_models/chat_models';

 const anthropicChatModels: Record<string, string>[] = [
+  {
+    displayName: 'Claude 4 Opus',
+    key: 'claude-opus-4-20250514',
+  },
+  {
+    displayName: 'Claude 4 Sonnet',
+    key: 'claude-sonnet-4-20250514',
+  },
  {
    displayName: 'Claude 3.7 Sonnet',
    key: 'claude-3-7-sonnet-20250219',
@ -29,10 +37,6 @@ const anthropicChatModels: Record<string, string>[] = [
    displayName: 'Claude 3 Opus',
    key: 'claude-3-opus-20240229',
  },
-  {
-    displayName: 'Claude 3 Sonnet',
-    key: 'claude-3-sonnet-20240229',
-  },
  {
    displayName: 'Claude 3 Haiku',
    key: 'claude-3-haiku-20240307',
@ -53,7 +57,7 @@ export const loadAnthropicChatModels = async () => {
        model: new ChatAnthropic({
          apiKey: anthropicApiKey,
          modelName: model.key,
-          temperature: 0.7,
+          // temperature: 0.7,
        }) as unknown as BaseChatModel,
      };
    });
--- a/src/lib/providers/deepseek.ts
+++ b/src/lib/providers/deepseek.ts
@ -33,7 +33,7 @@ export const loadDeepseekChatModels = async () => {
        model: new ChatOpenAI({
          openAIApiKey: deepseekApiKey,
          modelName: model.key,
-          temperature: 0.7,
+          // temperature: 0.7,
          configuration: {
            baseURL: 'https://api.deepseek.com',
          },
--- a/src/lib/providers/gemini.ts
+++ b/src/lib/providers/gemini.ts
@ -75,8 +75,8 @@ export const loadGeminiChatModels = async () => {
        displayName: model.displayName,
        model: new ChatGoogleGenerativeAI({
          apiKey: geminiApiKey,
-          modelName: model.key,
-          temperature: 0.7,
+          model: model.key,
+          // temperature: 0.7,
        }) as unknown as BaseChatModel,
      };
    });
--- a/src/lib/providers/groq.ts
+++ b/src/lib/providers/groq.ts
@ -1,40 +1,112 @@
-import { ChatOpenAI } from '@langchain/openai';
+import { ChatGroq } from '@langchain/groq';
 import { getGroqApiKey } from '../config';
 import { ChatModel } from '.';
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';

 export const PROVIDER_INFO = {
  key: 'groq',
  displayName: 'Groq',
 };

-import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+interface GroqModel {
+  id: string;
+  object: string;
+  created: number;
+  owned_by: string;
+  active: boolean;
+  context_window: number;
+  max_completion_tokens: number;
+}

-export const loadGroqChatModels = async () => {
-  const groqApiKey = getGroqApiKey();
-  if (!groqApiKey) return {};
+interface GroqModelsResponse {
+  object: string;
+  data: GroqModel[];
+}

+const generateDisplayName = (modelId: string, ownedBy: string): string => {
+  // Handle special cases for better display names
+  const modelMap: Record<string, string> = {
+    'gemma2-9b-it': 'Gemma2 9B IT',
+    'llama-3.3-70b-versatile': 'Llama 3.3 70B Versatile',
+    'llama-3.1-8b-instant': 'Llama 3.1 8B Instant',
+    'llama3-70b-8192': 'Llama3 70B 8192',
+    'llama3-8b-8192': 'Llama3 8B 8192',
+    'mixtral-8x7b-32768': 'Mixtral 8x7B 32768',
+    'qwen-qwq-32b': 'Qwen QWQ 32B',
+    'mistral-saba-24b': 'Mistral Saba 24B',
+    'deepseek-r1-distill-llama-70b': 'DeepSeek R1 Distill Llama 70B',
+    'deepseek-r1-distill-qwen-32b': 'DeepSeek R1 Distill Qwen 32B',
+  };
+
+  // Return mapped name if available
+  if (modelMap[modelId]) {
+    return modelMap[modelId];
+  }
+
+  // Generate display name from model ID
+  let displayName = modelId
+    .replace(/[-_]/g, ' ')
+    .split(' ')
+    .map((word) => word.charAt(0).toUpperCase() + word.slice(1))
+    .join(' ');
+
+  // Add owner info for certain models
+  if (modelId.includes('meta-llama/')) {
+    displayName = displayName.replace('Meta Llama/', '');
+  }
+
+  return displayName;
+};
+
+const fetchGroqModels = async (apiKey: string): Promise<GroqModel[]> => {
  try {
-    const res = await fetch('https://api.groq.com/openai/v1/models', {
-      method: 'GET',
+    const response = await fetch('https://api.groq.com/openai/v1/models', {
      headers: {
-        Authorization: `bearer ${groqApiKey}`,
+        Authorization: `Bearer ${apiKey}`,
        'Content-Type': 'application/json',
      },
    });

-    const groqChatModels = (await res.json()).data;
+    if (!response.ok) {
+      throw new Error(
+        `Failed to fetch models: ${response.status} ${response.statusText}`,
+      );
+    }
+
+    const data: GroqModelsResponse = await response.json();
+
+    // Filter for active chat completion models (exclude audio/whisper models)
+    return data.data.filter(
+      (model) =>
+        model.active &&
+        !model.id.includes('whisper') &&
+        !model.id.includes('tts') &&
+        !model.id.includes('guard') &&
+        !model.id.includes('prompt-guard'),
+    );
+  } catch (error) {
+    console.error('Error fetching Groq models:', error);
+    return [];
+  }
+};
+
+export const loadGroqChatModels = async () => {
+  const groqApiKey = getGroqApiKey();
+
+  if (!groqApiKey) return {};
+
+  try {
    const chatModels: Record<string, ChatModel> = {};

-    groqChatModels.forEach((model: any) => {
+    // Fetch available models from Groq API
+    const availableModels = await fetchGroqModels(groqApiKey);
+
+    availableModels.forEach((model) => {
      chatModels[model.id] = {
-        displayName: model.id,
-        model: new ChatOpenAI({
-          openAIApiKey: groqApiKey,
-          modelName: model.id,
-          temperature: 0.7,
-          configuration: {
-            baseURL: 'https://api.groq.com/openai/v1',
-          },
+        displayName: generateDisplayName(model.id, model.owned_by),
+        model: new ChatGroq({
+          apiKey: groqApiKey,
+          model: model.id,
        }) as unknown as BaseChatModel,
      };
    });
--- a/src/lib/providers/index.ts
+++ b/src/lib/providers/index.ts
@ -10,6 +10,7 @@ import {
  getCustomOpenaiApiKey,
  getCustomOpenaiApiUrl,
  getCustomOpenaiModelName,
+  getHiddenModels,
 } from '../config';
 import { ChatOpenAI } from '@langchain/openai';
 import {
@ -45,6 +46,10 @@ import {
  loadLMStudioEmbeddingsModels,
  PROVIDER_INFO as LMStudioInfo,
 } from './lmstudio';
+import {
+  loadOpenrouterChatModels,
+  PROVIDER_INFO as OpenRouterInfo,
+} from './openrouter';

 export const PROVIDER_METADATA = {
  openai: OpenAIInfo,
@ -56,6 +61,7 @@ export const PROVIDER_METADATA = {
  deepseek: DeepseekInfo,
  aimlapi: AimlApiInfo,
  lmstudio: LMStudioInfo,
+  openrouter: OpenRouterInfo,
  custom_openai: {
    key: 'custom_openai',
    displayName: 'Custom OpenAI',
@ -84,6 +90,7 @@ export const chatModelProviders: Record<
  deepseek: loadDeepseekChatModels,
  aimlapi: loadAimlApiChatModels,
  lmstudio: loadLMStudioChatModels,
+  openrouter: loadOpenrouterChatModels,
 };

 export const embeddingModelProviders: Record<
@ -98,13 +105,23 @@ export const embeddingModelProviders: Record<
  lmstudio: loadLMStudioEmbeddingsModels,
 };

-export const getAvailableChatModelProviders = async () => {
+export const getAvailableChatModelProviders = async (
+  options: { includeHidden?: boolean } = {},
+) => {
+  const { includeHidden = false } = options;
  const models: Record<string, Record<string, ChatModel>> = {};

  for (const provider in chatModelProviders) {
    const providerModels = await chatModelProviders[provider]();
    if (Object.keys(providerModels).length > 0) {
-      models[provider] = providerModels;
+      // Sort models alphabetically by their keys
+      const sortedModels: Record<string, ChatModel> = {};
+      Object.keys(providerModels)
+        .sort()
+        .forEach((key) => {
+          sortedModels[key] = providerModels[key];
+        });
+      models[provider] = sortedModels;
    }
  }

@ -112,34 +129,79 @@ export const getAvailableChatModelProviders = async () => {
  const customOpenAiApiUrl = getCustomOpenaiApiUrl();
  const customOpenAiModelName = getCustomOpenaiModelName();

+  // Only add custom_openai provider if all required fields are configured
+  if (customOpenAiApiKey && customOpenAiApiUrl && customOpenAiModelName) {
    models['custom_openai'] = {
-    ...(customOpenAiApiKey && customOpenAiApiUrl && customOpenAiModelName
-      ? {
      [customOpenAiModelName]: {
        displayName: customOpenAiModelName,
        model: new ChatOpenAI({
          openAIApiKey: customOpenAiApiKey,
          modelName: customOpenAiModelName,
-              temperature: 0.7,
+          // temperature: 0.7,
          configuration: {
            baseURL: customOpenAiApiUrl,
          },
        }) as unknown as BaseChatModel,
      },
-        }
-      : {}),
    };
+  }
+
+  // Filter out hidden models if includeHidden is false
+  if (!includeHidden) {
+    const hiddenModels = getHiddenModels();
+    if (hiddenModels.length > 0) {
+      for (const provider in models) {
+        for (const modelKey in models[provider]) {
+          if (hiddenModels.includes(modelKey)) {
+            delete models[provider][modelKey];
+          }
+        }
+        // Remove provider if all models are hidden
+        if (Object.keys(models[provider]).length === 0) {
+          delete models[provider];
+        }
+      }
+    }
+  }

  return models;
 };

-export const getAvailableEmbeddingModelProviders = async () => {
+export const getAvailableEmbeddingModelProviders = async (
+  options: { includeHidden?: boolean } = {},
+) => {
+  const { includeHidden = false } = options;
  const models: Record<string, Record<string, EmbeddingModel>> = {};

  for (const provider in embeddingModelProviders) {
    const providerModels = await embeddingModelProviders[provider]();
    if (Object.keys(providerModels).length > 0) {
-      models[provider] = providerModels;
+      // Sort embedding models alphabetically by their keys
+      const sortedModels: Record<string, EmbeddingModel> = {};
+      Object.keys(providerModels)
+        .sort()
+        .forEach((key) => {
+          sortedModels[key] = providerModels[key];
+        });
+      models[provider] = sortedModels;
+    }
+  }
+
+  // Filter out hidden models if includeHidden is false
+  if (!includeHidden) {
+    const hiddenModels = getHiddenModels();
+    if (hiddenModels.length > 0) {
+      for (const provider in models) {
+        for (const modelKey in models[provider]) {
+          if (hiddenModels.includes(modelKey)) {
+            delete models[provider][modelKey];
+          }
+        }
+        // Remove provider if all models are hidden
+        if (Object.keys(models[provider]).length === 0) {
+          delete models[provider];
+        }
+      }
    }
  }

--- a/src/lib/providers/lmstudio.ts
+++ b/src/lib/providers/lmstudio.ts
@ -52,7 +52,7 @@ export const loadLMStudioChatModels = async () => {
            baseURL: ensureV1Endpoint(endpoint),
          },
          modelName: model.id,
-          temperature: 0.7,
+          // temperature: 0.7,
          streaming: true,
          maxRetries: 3,
        }) as unknown as BaseChatModel,
--- a/src/lib/providers/ollama.ts
+++ b/src/lib/providers/ollama.ts
@ -6,8 +6,8 @@ export const PROVIDER_INFO = {
  key: 'ollama',
  displayName: 'Ollama',
 };
-import { ChatOllama } from '@langchain/community/chat_models/ollama';
-import { OllamaEmbeddings } from '@langchain/community/embeddings/ollama';
+import { ChatOllama } from '@langchain/ollama';
+import { OllamaEmbeddings } from '@langchain/ollama';

 export const loadOllamaChatModels = async () => {
  const ollamaApiEndpoint = getOllamaApiEndpoint();
@ -31,7 +31,7 @@ export const loadOllamaChatModels = async () => {
        model: new ChatOllama({
          baseUrl: ollamaApiEndpoint,
          model: model.model,
-          temperature: 0.7,
+          // temperature: 0.7,
          keepAlive: getKeepAlive(),
        }),
      };
--- a/src/lib/providers/openai.ts
+++ b/src/lib/providers/openai.ts
@ -69,7 +69,7 @@ export const loadOpenAIChatModels = async () => {
        model: new ChatOpenAI({
          openAIApiKey: openaiApiKey,
          modelName: model.key,
-          temperature: 0.7,
+          // temperature: 0.7,
        }) as unknown as BaseChatModel,
      };
    });
--- a/src/lib/providers/openrouter.ts
+++ b/src/lib/providers/openrouter.ts
@ -0,0 +1,65 @@
+export const PROVIDER_INFO = {
+  key: 'openrouter',
+  displayName: 'OpenRouter',
+};
+import { ChatOpenAI } from '@langchain/openai';
+import { getOpenrouterApiKey } from '../config';
+import { ChatModel } from '.';
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+
+let openrouterChatModels: Record<string, string>[] = [];
+
+async function fetchModelList(): Promise<void> {
+  try {
+    const response = await fetch('https://openrouter.ai/api/v1/models', {
+      method: 'GET',
+      headers: {
+        'Content-Type': 'application/json',
+      },
+    });
+
+    if (!response.ok) {
+      throw new Error(`API request failed with status: ${response.status}`);
+    }
+
+    const data = await response.json();
+
+    openrouterChatModels = data.data.map((model: any) => ({
+      displayName: model.name,
+      key: model.id,
+    }));
+  } catch (error) {
+    console.error('Error fetching models:', error);
+  }
+}
+
+export const loadOpenrouterChatModels = async () => {
+  await fetchModelList();
+
+  const openrouterApikey = getOpenrouterApiKey();
+
+  if (!openrouterApikey) return {};
+
+  try {
+    const chatModels: Record<string, ChatModel> = {};
+
+    openrouterChatModels.forEach((model) => {
+      chatModels[model.key] = {
+        displayName: model.displayName,
+        model: new ChatOpenAI({
+          openAIApiKey: openrouterApikey,
+          modelName: model.key,
+          temperature: 0.7,
+          configuration: {
+            baseURL: 'https://openrouter.ai/api/v1',
+          },
+        }) as unknown as BaseChatModel,
+      };
+    });
+
+    return chatModels;
+  } catch (err) {
+    console.error(`Error loading Openrouter models: ${err}`);
+    return {};
+  }
+};
--- a/src/lib/search/agentSearch.ts
+++ b/src/lib/search/agentSearch.ts
@ -0,0 +1,311 @@
+import { Embeddings } from '@langchain/core/embeddings';
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import {
+  BaseMessage,
+  HumanMessage,
+  SystemMessage,
+} from '@langchain/core/messages';
+import {
+  BaseLangGraphError,
+  END,
+  GraphRecursionError,
+  MemorySaver,
+  START,
+  StateGraph,
+} from '@langchain/langgraph';
+import { EventEmitter } from 'events';
+import {
+  AgentState,
+  WebSearchAgent,
+  AnalyzerAgent,
+  SynthesizerAgent,
+  TaskManagerAgent,
+  FileSearchAgent,
+  ContentRouterAgent,
+  URLSummarizationAgent,
+} from '../agents';
+
+/**
+ * Agent Search class implementing LangGraph Supervisor pattern
+ */
+export class AgentSearch {
+  private llm: BaseChatModel;
+  private embeddings: Embeddings;
+  private checkpointer: MemorySaver;
+  private signal: AbortSignal;
+  private taskManagerAgent: TaskManagerAgent;
+  private webSearchAgent: WebSearchAgent;
+  private analyzerAgent: AnalyzerAgent;
+  private synthesizerAgent: SynthesizerAgent;
+  private fileSearchAgent: FileSearchAgent;
+  private contentRouterAgent: ContentRouterAgent;
+  private urlSummarizationAgent: URLSummarizationAgent;
+  private emitter: EventEmitter;
+  private focusMode: string;
+
+  constructor(
+    llm: BaseChatModel,
+    embeddings: Embeddings,
+    emitter: EventEmitter,
+    systemInstructions: string = '',
+    personaInstructions: string = '',
+    signal: AbortSignal,
+    focusMode: string = 'webSearch',
+  ) {
+    this.llm = llm;
+    this.embeddings = embeddings;
+    this.checkpointer = new MemorySaver();
+    this.signal = signal;
+    this.emitter = emitter;
+    this.focusMode = focusMode;
+
+    // Initialize agents
+    this.taskManagerAgent = new TaskManagerAgent(
+      llm,
+      emitter,
+      systemInstructions,
+      signal,
+    );
+    this.webSearchAgent = new WebSearchAgent(
+      llm,
+      emitter,
+      systemInstructions,
+      signal,
+      embeddings,
+    );
+    this.analyzerAgent = new AnalyzerAgent(
+      llm,
+      emitter,
+      systemInstructions,
+      signal,
+    );
+    this.synthesizerAgent = new SynthesizerAgent(
+      llm,
+      emitter,
+      personaInstructions,
+      signal,
+    );
+    this.fileSearchAgent = new FileSearchAgent(
+      llm,
+      emitter,
+      systemInstructions,
+      signal,
+      embeddings,
+    );
+    this.contentRouterAgent = new ContentRouterAgent(
+      llm,
+      emitter,
+      systemInstructions,
+      signal,
+    );
+    this.urlSummarizationAgent = new URLSummarizationAgent(
+      llm,
+      emitter,
+      systemInstructions,
+      signal,
+    );
+  }
+
+  /**
+   * Create and compile the agent workflow graph
+   */
+  private createWorkflow() {
+    const workflow = new StateGraph(AgentState)
+      .addNode(
+        'url_summarization',
+        this.urlSummarizationAgent.execute.bind(this.urlSummarizationAgent),
+        {
+          ends: ['task_manager', 'analyzer'],
+        },
+      )
+      .addNode(
+        'task_manager',
+        this.taskManagerAgent.execute.bind(this.taskManagerAgent),
+        {
+          ends: ['content_router', 'analyzer'],
+        },
+      )
+      .addNode(
+        'content_router',
+        this.contentRouterAgent.execute.bind(this.contentRouterAgent),
+        {
+          ends: ['file_search', 'web_search', 'analyzer'],
+        },
+      )
+      .addNode(
+        'file_search',
+        this.fileSearchAgent.execute.bind(this.fileSearchAgent),
+        {
+          ends: ['analyzer'],
+        },
+      )
+      .addNode(
+        'web_search',
+        this.webSearchAgent.execute.bind(this.webSearchAgent),
+        {
+          ends: ['analyzer'],
+        },
+      )
+      .addNode(
+        'analyzer',
+        this.analyzerAgent.execute.bind(this.analyzerAgent),
+        {
+          ends: ['url_summarization', 'task_manager', 'synthesizer'],
+        },
+      )
+      .addNode(
+        'synthesizer',
+        this.synthesizerAgent.execute.bind(this.synthesizerAgent),
+        {
+          ends: [END],
+        },
+      )
+      .addEdge(START, 'analyzer');
+
+    return workflow.compile({ checkpointer: this.checkpointer });
+  }
+
+  /**
+   * Execute the agent search workflow
+   */
+  async searchAndAnswer(
+    query: string,
+    history: BaseMessage[] = [],
+    fileIds: string[] = [],
+  ) {
+    const workflow = this.createWorkflow();
+
+    const initialState = {
+      messages: [...history, new HumanMessage(query)],
+      query,
+      fileIds,
+      focusMode: this.focusMode,
+    };
+
+    const threadId = `agent_search_${Date.now()}`;
+    const config = {
+      configurable: { thread_id: threadId },
+      recursionLimit: 18,
+      signal: this.signal,
+    };
+
+    try {
+      const result = await workflow.invoke(initialState, config);
+    } catch (error: any) {
+      if (error instanceof GraphRecursionError) {
+        console.warn(
+          'Graph recursion limit reached, attempting best-effort synthesis with gathered information',
+        );
+
+        // Emit agent action to explain what happened
+        this.emitter.emit(
+          'data',
+          JSON.stringify({
+            type: 'agent_action',
+            data: {
+              action: 'recursion_limit_recovery',
+              message:
+                'Search process reached complexity limits. Attempting to provide best-effort response with gathered information.',
+              details:
+                'The agent workflow exceeded the maximum number of steps allowed. Recovering by synthesizing available data.',
+            },
+          }),
+        );
+
+        try {
+          // Get the latest state from the checkpointer to access gathered information
+          const latestState = await workflow.getState({
+            configurable: { thread_id: threadId },
+          });
+
+          if (latestState && latestState.values) {
+            // Create emergency synthesis state using gathered information
+            const stateValues = latestState.values;
+            const emergencyState = {
+              messages: stateValues.messages || initialState.messages,
+              query: stateValues.query || initialState.query,
+              relevantDocuments: stateValues.relevantDocuments || [],
+              bannedSummaryUrls: stateValues.bannedSummaryUrls || [],
+              bannedPreviewUrls: stateValues.bannedPreviewUrls || [],
+              searchInstructionHistory:
+                stateValues.searchInstructionHistory || [],
+              searchInstructions: stateValues.searchInstructions || '',
+              next: 'synthesizer',
+              analysis: stateValues.analysis || '',
+              fullAnalysisAttempts: stateValues.fullAnalysisAttempts || 0,
+              tasks: stateValues.tasks || [],
+              currentTaskIndex: stateValues.currentTaskIndex || 0,
+              originalQuery:
+                stateValues.originalQuery ||
+                stateValues.query ||
+                initialState.query,
+              fileIds: stateValues.fileIds || initialState.fileIds,
+              focusMode: stateValues.focusMode || initialState.focusMode,
+              urlsToSummarize: stateValues.urlsToSummarize || [],
+              summarizationIntent: stateValues.summarizationIntent || '',
+              recursionLimitReached: true,
+            };
+
+            const documentsCount =
+              emergencyState.relevantDocuments?.length || 0;
+            console.log(
+              `Attempting emergency synthesis with ${documentsCount} gathered documents`,
+            );
+
+            // Emit detailed agent action about the recovery attempt
+            this.emitter.emit(
+              'data',
+              JSON.stringify({
+                type: 'agent_action',
+                data: {
+                  action: 'emergency_synthesis',
+                  message: `Proceeding with available information: ${documentsCount} documents gathered${emergencyState.analysis ? ', analysis available' : ''}`,
+                  details: `Recovered state contains: ${documentsCount} relevant documents, ${emergencyState.searchInstructionHistory?.length || 0} search attempts, ${emergencyState.analysis ? 'analysis data' : 'no analysis'}`,
+                },
+              }),
+            );
+
+            // Only proceed with synthesis if we have some useful information
+            if (documentsCount > 0 || emergencyState.analysis) {
+              await this.synthesizerAgent.execute(emergencyState);
+            } else {
+              // If we don't have any gathered information, provide a helpful message
+              this.emitter.emit(
+                'data',
+                JSON.stringify({
+                  type: 'response',
+                  data: "⚠️ **Search Process Incomplete** - The search process reached complexity limits before gathering sufficient information to provide a meaningful response. Please try:\n\n- Using more specific keywords\n- Breaking your question into smaller parts\n- Rephrasing your query to be more focused\n\nI apologize that I couldn't provide the information you were looking for.",
+                }),
+              );
+              this.emitter.emit('end');
+            }
+          } else {
+            // Fallback if we can't retrieve state
+            this.emitter.emit(
+              'data',
+              JSON.stringify({
+                type: 'response',
+                data: '⚠️ **Limited Information Available** - The search process encountered complexity limits and was unable to gather sufficient information. Please try rephrasing your question or breaking it into smaller, more specific parts.',
+              }),
+            );
+            this.emitter.emit('end');
+          }
+        } catch (synthError) {
+          console.error('Emergency synthesis failed:', synthError);
+          this.emitter.emit(
+            'data',
+            JSON.stringify({
+              type: 'response',
+              data: '⚠️ **Search Process Interrupted** - The search encountered complexity limits and could not complete successfully. Please try a simpler query or break your question into smaller parts.',
+            }),
+          );
+          this.emitter.emit('end');
+        }
+      } else if (error.name === 'AbortError') {
+        console.warn('Agent search was aborted:', error.message);
+      } else {
+        console.error('Unexpected error during agent search:', error);
+      }
+    }
+  }
+}
--- a/src/lib/search/index.ts
+++ b/src/lib/search/index.ts
@ -1,6 +1,9 @@
 import MetaSearchAgent from '@/lib/search/metaSearchAgent';
+import SpeedSearchAgent from '@/lib/search/speedSearch';
 import prompts from '../prompts';

+export { default as SpeedSearchAgent } from './speedSearch';
+
 export const searchHandlers: Record<string, MetaSearchAgent> = {
  webSearch: new MetaSearchAgent({
    activeEngines: [],
@ -13,26 +16,35 @@ export const searchHandlers: Record<string, MetaSearchAgent> = {
  }),
  academicSearch: new MetaSearchAgent({
    activeEngines: ['arxiv', 'google scholar', 'pubmed'],
-    queryGeneratorPrompt: prompts.academicSearchRetrieverPrompt,
-    responsePrompt: prompts.academicSearchResponsePrompt,
+    queryGeneratorPrompt: prompts.webSearchRetrieverPrompt,
+    responsePrompt: prompts.webSearchResponsePrompt,
    rerank: true,
    rerankThreshold: 0,
    searchWeb: true,
    summarizer: false,
  }),
-  writingAssistant: new MetaSearchAgent({
+  localResearch: new MetaSearchAgent({
    activeEngines: [],
    queryGeneratorPrompt: '',
-    responsePrompt: prompts.writingAssistantPrompt,
+    responsePrompt: prompts.localResearchPrompt,
    rerank: true,
    rerankThreshold: 0,
    searchWeb: false,
    summarizer: false,
  }),
+  chat: new MetaSearchAgent({
+    activeEngines: [],
+    queryGeneratorPrompt: '',
+    responsePrompt: prompts.chatPrompt,
+    rerank: false,
+    rerankThreshold: 0,
+    searchWeb: false,
+    summarizer: false,
+  }),
  wolframAlphaSearch: new MetaSearchAgent({
    activeEngines: ['wolframalpha'],
-    queryGeneratorPrompt: prompts.wolframAlphaSearchRetrieverPrompt,
-    responsePrompt: prompts.wolframAlphaSearchResponsePrompt,
+    queryGeneratorPrompt: prompts.webSearchRetrieverPrompt,
+    responsePrompt: prompts.webSearchResponsePrompt,
    rerank: false,
    rerankThreshold: 0,
    searchWeb: true,
@ -40,20 +52,21 @@ export const searchHandlers: Record<string, MetaSearchAgent> = {
  }),
  youtubeSearch: new MetaSearchAgent({
    activeEngines: ['youtube'],
-    queryGeneratorPrompt: prompts.youtubeSearchRetrieverPrompt,
-    responsePrompt: prompts.youtubeSearchResponsePrompt,
+    queryGeneratorPrompt: prompts.webSearchRetrieverPrompt,
+    responsePrompt: prompts.webSearchResponsePrompt,
    rerank: true,
    rerankThreshold: 0.3,
    searchWeb: true,
    summarizer: false,
  }),
  redditSearch: new MetaSearchAgent({
-    activeEngines: ['reddit'],
-    queryGeneratorPrompt: prompts.redditSearchRetrieverPrompt,
-    responsePrompt: prompts.redditSearchResponsePrompt,
+    activeEngines: [],
+    queryGeneratorPrompt: prompts.webSearchRetrieverPrompt,
+    responsePrompt: prompts.webSearchResponsePrompt,
    rerank: true,
    rerankThreshold: 0.3,
    searchWeb: true,
    summarizer: false,
+    additionalSearchCriteria: "'site:reddit.com'",
  }),
 };
--- a/src/lib/search/metaSearchAgent.ts
+++ b/src/lib/search/metaSearchAgent.ts
@ -1,29 +1,9 @@
-import { ChatOpenAI } from '@langchain/openai';
-import type { BaseChatModel } from '@langchain/core/language_models/chat_models';
 import type { Embeddings } from '@langchain/core/embeddings';
-import {
-  ChatPromptTemplate,
-  MessagesPlaceholder,
-  PromptTemplate,
-} from '@langchain/core/prompts';
-import {
-  RunnableLambda,
-  RunnableMap,
-  RunnableSequence,
-} from '@langchain/core/runnables';
+import type { BaseChatModel } from '@langchain/core/language_models/chat_models';
 import { BaseMessage } from '@langchain/core/messages';
-import { StringOutputParser } from '@langchain/core/output_parsers';
-import LineListOutputParser from '../outputParsers/listLineOutputParser';
-import LineOutputParser from '../outputParsers/lineOutputParser';
-import { getDocumentsFromLinks } from '../utils/documents';
-import { Document } from 'langchain/document';
-import { searchSearxng } from '../searxng';
-import path from 'node:path';
-import fs from 'node:fs';
-import computeSimilarity from '../utils/computeSimilarity';
-import formatChatHistoryAsString from '../utils/formatHistory';
 import eventEmitter from 'events';
-import { StreamEvent } from '@langchain/core/tracers/log_stream';
+import { AgentSearch } from './agentSearch';
+import SpeedSearchAgent from './speedSearch';

 export interface MetaSearchAgentType {
  searchAndAnswer: (
@ -31,9 +11,12 @@ export interface MetaSearchAgentType {
    history: BaseMessage[],
    llm: BaseChatModel,
    embeddings: Embeddings,
-    optimizationMode: 'speed' | 'balanced' | 'quality',
+    optimizationMode: 'speed' | 'agent',
    fileIds: string[],
    systemInstructions: string,
+    signal: AbortSignal,
+    personaInstructions?: string,
+    focusMode?: string,
  ) => Promise<eventEmitter>;
 }

@ -45,456 +28,102 @@ interface Config {
  queryGeneratorPrompt: string;
  responsePrompt: string;
  activeEngines: string[];
+  additionalSearchCriteria?: string;
 }

-type BasicChainInput = {
-  chat_history: BaseMessage[];
-  query: string;
-};
-
 class MetaSearchAgent implements MetaSearchAgentType {
  private config: Config;
-  private strParser = new StringOutputParser();

  constructor(config: Config) {
    this.config = config;
  }

-  private async createSearchRetrieverChain(llm: BaseChatModel) {
-    (llm as unknown as ChatOpenAI).temperature = 0;
-
-    return RunnableSequence.from([
-      PromptTemplate.fromTemplate(this.config.queryGeneratorPrompt),
-      llm,
-      this.strParser,
-      RunnableLambda.from(async (input: string) => {
-        const linksOutputParser = new LineListOutputParser({
-          key: 'links',
-        });
-
-        const questionOutputParser = new LineOutputParser({
-          key: 'question',
-        });
-
-        const links = await linksOutputParser.parse(input);
-        let question = this.config.summarizer
-          ? await questionOutputParser.parse(input)
-          : input;
-
-        if (question === 'not_needed') {
-          return { query: '', docs: [] };
-        }
-
-        if (links.length > 0) {
-          if (question.length === 0) {
-            question = 'summarize';
-          }
-
-          let docs: Document[] = [];
-
-          const linkDocs = await getDocumentsFromLinks({ links });
-
-          const docGroups: Document[] = [];
-
-          linkDocs.map((doc) => {
-            const URLDocExists = docGroups.find(
-              (d) =>
-                d.metadata.url === doc.metadata.url &&
-                d.metadata.totalDocs < 10,
-            );
-
-            if (!URLDocExists) {
-              docGroups.push({
-                ...doc,
-                metadata: {
-                  ...doc.metadata,
-                  totalDocs: 1,
-                },
-              });
-            }
-
-            const docIndex = docGroups.findIndex(
-              (d) =>
-                d.metadata.url === doc.metadata.url &&
-                d.metadata.totalDocs < 10,
-            );
-
-            if (docIndex !== -1) {
-              docGroups[docIndex].pageContent =
-                docGroups[docIndex].pageContent + `\n\n` + doc.pageContent;
-              docGroups[docIndex].metadata.totalDocs += 1;
-            }
-          });
-
-          await Promise.all(
-            docGroups.map(async (doc) => {
-              const res = await llm.invoke(`
-            You are a web search summarizer, tasked with summarizing a piece of text retrieved from a web search. Your job is to summarize the 
-            text into a detailed, 2-4 paragraph explanation that captures the main ideas and provides a comprehensive answer to the query.
-            If the query is \"summarize\", you should provide a detailed summary of the text. If the query is a specific question, you should answer it in the summary.
-            
-            - **Journalistic tone**: The summary should sound professional and journalistic, not too casual or vague.
-            - **Thorough and detailed**: Ensure that every key point from the text is captured and that the summary directly answers the query.
-            - **Not too lengthy, but detailed**: The summary should be informative but not excessively long. Focus on providing detailed information in a concise format.
-
-            The text will be shared inside the \`text\` XML tag, and the query inside the \`query\` XML tag.
-
-            <example>
-            1. \`<text>
-            Docker is a set of platform-as-a-service products that use OS-level virtualization to deliver software in packages called containers. 
-            It was first released in 2013 and is developed by Docker, Inc. Docker is designed to make it easier to create, deploy, and run applications 
-            by using containers.
-            </text>
-
-            <query>
-            What is Docker and how does it work?
-            </query>
-
-            Response:
-            Docker is a revolutionary platform-as-a-service product developed by Docker, Inc., that uses container technology to make application 
-            deployment more efficient. It allows developers to package their software with all necessary dependencies, making it easier to run in 
-            any environment. Released in 2013, Docker has transformed the way applications are built, deployed, and managed.
-            \`
-            2. \`<text>
-            The theory of relativity, or simply relativity, encompasses two interrelated theories of Albert Einstein: special relativity and general
-            relativity. However, the word "relativity" is sometimes used in reference to Galilean invariance. The term "theory of relativity" was based
-            on the expression "relative theory" used by Max Planck in 1906. The theory of relativity usually encompasses two interrelated theories by
-            Albert Einstein: special relativity and general relativity. Special relativity applies to all physical phenomena in the absence of gravity.
-            General relativity explains the law of gravitation and its relation to other forces of nature. It applies to the cosmological and astrophysical
-            realm, including astronomy.
-            </text>
-
-            <query>
-            summarize
-            </query>
-
-            Response:
-            The theory of relativity, developed by Albert Einstein, encompasses two main theories: special relativity and general relativity. Special
-            relativity applies to all physical phenomena in the absence of gravity, while general relativity explains the law of gravitation and its
-            relation to other forces of nature. The theory of relativity is based on the concept of "relative theory," as introduced by Max Planck in
-            1906. It is a fundamental theory in physics that has revolutionized our understanding of the universe.
-            \`
-            </example>
-
-            Everything below is the actual data you will be working with. Good luck!
-
-            <query>
-            ${question}
-            </query>
-
-            <text>
-            ${doc.pageContent}
-            </text>
-
-            Make sure to answer the query in the summary.
-          `);
-
-              const document = new Document({
-                pageContent: res.content as string,
-                metadata: {
-                  title: doc.metadata.title,
-                  url: doc.metadata.url,
-                },
-              });
-
-              docs.push(document);
-            }),
-          );
-
-          return { query: question, docs: docs };
-        } else {
-          question = question.replace(/<think>.*?<\/think>/g, '');
-
-          const res = await searchSearxng(question, {
-            language: 'en',
-            engines: this.config.activeEngines,
-          });
-
-          const documents = res.results.map(
-            (result) =>
-              new Document({
-                pageContent:
-                  result.content ||
-                  (this.config.activeEngines.includes('youtube')
-                    ? result.title
-                    : '') /* Todo: Implement transcript grabbing using Youtubei (source: https://www.npmjs.com/package/youtubei) */,
-                metadata: {
-                  title: result.title,
-                  url: result.url,
-                  ...(result.img_src && { img_src: result.img_src }),
-                },
-              }),
-          );
-
-          return { query: question, docs: documents };
-        }
-      }),
-    ]);
-  }
-
-  private async createAnsweringChain(
+  /**
+   * Execute agent workflow asynchronously with proper streaming support
+   */
+  private async executeAgentWorkflow(
    llm: BaseChatModel,
-    fileIds: string[],
    embeddings: Embeddings,
-    optimizationMode: 'speed' | 'balanced' | 'quality',
-    systemInstructions: string,
-  ) {
-    return RunnableSequence.from([
-      RunnableMap.from({
-        systemInstructions: () => systemInstructions,
-        query: (input: BasicChainInput) => input.query,
-        chat_history: (input: BasicChainInput) => input.chat_history,
-        date: () => new Date().toISOString(),
-        context: RunnableLambda.from(async (input: BasicChainInput) => {
-          const processedHistory = formatChatHistoryAsString(
-            input.chat_history,
-          );
-
-          let docs: Document[] | null = null;
-          let query = input.query;
-
-          if (this.config.searchWeb) {
-            const searchRetrieverChain =
-              await this.createSearchRetrieverChain(llm);
-
-            const searchRetrieverResult = await searchRetrieverChain.invoke({
-              chat_history: processedHistory,
-              query,
-            });
-
-            query = searchRetrieverResult.query;
-            docs = searchRetrieverResult.docs;
-          }
-
-          const sortedDocs = await this.rerankDocs(
-            query,
-            docs ?? [],
-            fileIds,
-            embeddings,
-            optimizationMode,
-          );
-
-          return sortedDocs;
-        })
-          .withConfig({
-            runName: 'FinalSourceRetriever',
-          })
-          .pipe(this.processDocs),
-      }),
-      ChatPromptTemplate.fromMessages([
-        ['system', this.config.responsePrompt],
-        new MessagesPlaceholder('chat_history'),
-        ['user', '{query}'],
-      ]),
-      llm,
-      this.strParser,
-    ]).withConfig({
-      runName: 'FinalResponseGenerator',
-    });
-  }
-
-  private async rerankDocs(
-    query: string,
-    docs: Document[],
-    fileIds: string[],
-    embeddings: Embeddings,
-    optimizationMode: 'speed' | 'balanced' | 'quality',
-  ) {
-    if (docs.length === 0 && fileIds.length === 0) {
-      return docs;
-    }
-
-    const filesData = fileIds
-      .map((file) => {
-        const filePath = path.join(process.cwd(), 'uploads', file);
-
-        const contentPath = filePath + '-extracted.json';
-        const embeddingsPath = filePath + '-embeddings.json';
-
-        const content = JSON.parse(fs.readFileSync(contentPath, 'utf8'));
-        const embeddings = JSON.parse(fs.readFileSync(embeddingsPath, 'utf8'));
-
-        const fileSimilaritySearchObject = content.contents.map(
-          (c: string, i: number) => {
-            return {
-              fileName: content.title,
-              content: c,
-              embeddings: embeddings.embeddings[i],
-            };
-          },
-        );
-
-        return fileSimilaritySearchObject;
-      })
-      .flat();
-
-    if (query.toLocaleLowerCase() === 'summarize') {
-      return docs.slice(0, 15);
-    }
-
-    const docsWithContent = docs.filter(
-      (doc) => doc.pageContent && doc.pageContent.length > 0,
-    );
-
-    if (optimizationMode === 'speed' || this.config.rerank === false) {
-      if (filesData.length > 0) {
-        const [queryEmbedding] = await Promise.all([
-          embeddings.embedQuery(query),
-        ]);
-
-        const fileDocs = filesData.map((fileData) => {
-          return new Document({
-            pageContent: fileData.content,
-            metadata: {
-              title: fileData.fileName,
-              url: `File`,
-            },
-          });
-        });
-
-        const similarity = filesData.map((fileData, i) => {
-          const sim = computeSimilarity(queryEmbedding, fileData.embeddings);
-
-          return {
-            index: i,
-            similarity: sim,
-          };
-        });
-
-        let sortedDocs = similarity
-          .filter(
-            (sim) => sim.similarity > (this.config.rerankThreshold ?? 0.3),
-          )
-          .sort((a, b) => b.similarity - a.similarity)
-          .slice(0, 15)
-          .map((sim) => fileDocs[sim.index]);
-
-        sortedDocs =
-          docsWithContent.length > 0 ? sortedDocs.slice(0, 8) : sortedDocs;
-
-        return [
-          ...sortedDocs,
-          ...docsWithContent.slice(0, 15 - sortedDocs.length),
-        ];
-      } else {
-        return docsWithContent.slice(0, 15);
-      }
-    } else if (optimizationMode === 'balanced') {
-      const [docEmbeddings, queryEmbedding] = await Promise.all([
-        embeddings.embedDocuments(
-          docsWithContent.map((doc) => doc.pageContent),
-        ),
-        embeddings.embedQuery(query),
-      ]);
-
-      docsWithContent.push(
-        ...filesData.map((fileData) => {
-          return new Document({
-            pageContent: fileData.content,
-            metadata: {
-              title: fileData.fileName,
-              url: `File`,
-            },
-          });
-        }),
-      );
-
-      docEmbeddings.push(...filesData.map((fileData) => fileData.embeddings));
-
-      const similarity = docEmbeddings.map((docEmbedding, i) => {
-        const sim = computeSimilarity(queryEmbedding, docEmbedding);
-
-        return {
-          index: i,
-          similarity: sim,
-        };
-      });
-
-      const sortedDocs = similarity
-        .filter((sim) => sim.similarity > (this.config.rerankThreshold ?? 0.3))
-        .sort((a, b) => b.similarity - a.similarity)
-        .slice(0, 15)
-        .map((sim) => docsWithContent[sim.index]);
-
-      return sortedDocs;
-    }
-
-    return [];
-  }
-
-  private processDocs(docs: Document[]) {
-    return docs
-      .map(
-        (_, index) =>
-          `${index + 1}. ${docs[index].metadata.title} ${docs[index].pageContent}`,
-      )
-      .join('\n');
-  }
-
-  private async handleStream(
-    stream: AsyncGenerator<StreamEvent, any, any>,
    emitter: eventEmitter,
+    message: string,
+    history: BaseMessage[],
+    fileIds: string[],
+    systemInstructions: string,
+    personaInstructions: string,
+    signal: AbortSignal,
+    focusMode: string,
  ) {
-    for await (const event of stream) {
-      if (
-        event.event === 'on_chain_end' &&
-        event.name === 'FinalSourceRetriever'
-      ) {
-        ``;
-        emitter.emit(
-          'data',
-          JSON.stringify({ type: 'sources', data: event.data.output }),
+    try {
+      const agentSearch = new AgentSearch(
+        llm,
+        embeddings,
+        emitter,
+        systemInstructions,
+        personaInstructions,
+        signal,
+        focusMode,
      );
-      }
-      if (
-        event.event === 'on_chain_stream' &&
-        event.name === 'FinalResponseGenerator'
-      ) {
+
+      // Execute the agent workflow
+      await agentSearch.searchAndAnswer(message, history, fileIds);
+
+      // No need to emit end signals here since synthesizerAgent
+      // is now streaming in real-time and emits them
+    } catch (error) {
+      console.error('Agent search error:', error);
      emitter.emit(
-          'data',
-          JSON.stringify({ type: 'response', data: event.data.chunk }),
+        'error',
+        JSON.stringify({
+          data: `Agent search failed: ${error instanceof Error ? error.message : 'Unknown error'}`,
+        }),
      );
-      }
-      if (
-        event.event === 'on_chain_end' &&
-        event.name === 'FinalResponseGenerator'
-      ) {
      emitter.emit('end');
    }
  }
-  }

  async searchAndAnswer(
    message: string,
    history: BaseMessage[],
    llm: BaseChatModel,
    embeddings: Embeddings,
-    optimizationMode: 'speed' | 'balanced' | 'quality',
+    optimizationMode: 'speed' | 'agent',
    fileIds: string[],
    systemInstructions: string,
+    signal: AbortSignal,
+    personaInstructions?: string,
+    focusMode?: string,
  ) {
    const emitter = new eventEmitter();

-    const answeringChain = await this.createAnsweringChain(
+    // Branch to speed search if optimization mode is 'speed'
+    if (optimizationMode === 'speed') {
+      const speedSearchAgent = new SpeedSearchAgent(this.config);
+      return speedSearchAgent.searchAndAnswer(
+        message,
+        history,
        llm,
-      fileIds,
        embeddings,
-      optimizationMode,
        systemInstructions,
+        signal,
+        personaInstructions,
+        focusMode,
      );
+    }

-    const stream = answeringChain.streamEvents(
-      {
-        chat_history: history,
-        query: message,
-      },
-      {
-        version: 'v1',
-      },
+    // Execute agent workflow for 'agent' mode
+    this.executeAgentWorkflow(
+      llm,
+      embeddings,
+      emitter,
+      message,
+      history,
+      fileIds,
+      systemInstructions,
+      personaInstructions || '',
+      signal,
+      focusMode || 'webSearch',
    );

-    this.handleStream(stream, emitter);
-
    return emitter;
  }
 }
--- a/src/lib/search/speedSearch.ts
+++ b/src/lib/search/speedSearch.ts
@ -0,0 +1,560 @@
+import type { Embeddings } from '@langchain/core/embeddings';
+import type { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import { BaseMessage } from '@langchain/core/messages';
+import { StringOutputParser } from '@langchain/core/output_parsers';
+import {
+  ChatPromptTemplate,
+  MessagesPlaceholder,
+  PromptTemplate,
+} from '@langchain/core/prompts';
+import {
+  RunnableLambda,
+  RunnableMap,
+  RunnableSequence,
+} from '@langchain/core/runnables';
+import { StreamEvent } from '@langchain/core/tracers/log_stream';
+import { ChatOpenAI } from '@langchain/openai';
+import eventEmitter from 'events';
+import { Document } from 'langchain/document';
+import LineOutputParser from '../outputParsers/lineOutputParser';
+import LineListOutputParser from '../outputParsers/listLineOutputParser';
+import { searchSearxng } from '../searxng';
+import { formatDateForLLM } from '../utils';
+import { getDocumentsFromLinks } from '../utils/documents';
+import formatChatHistoryAsString from '../utils/formatHistory';
+import { getModelName } from '../utils/modelUtils';
+
+export interface SpeedSearchAgentType {
+  searchAndAnswer: (
+    message: string,
+    history: BaseMessage[],
+    llm: BaseChatModel,
+    embeddings: Embeddings,
+    systemInstructions: string,
+    signal: AbortSignal,
+    personaInstructions?: string,
+    focusMode?: string,
+  ) => Promise<eventEmitter>;
+}
+
+interface Config {
+  searchWeb: boolean;
+  rerank: boolean;
+  summarizer: boolean;
+  rerankThreshold: number;
+  queryGeneratorPrompt: string;
+  responsePrompt: string;
+  activeEngines: string[];
+  additionalSearchCriteria?: string;
+}
+
+type BasicChainInput = {
+  chat_history: BaseMessage[];
+  query: string;
+};
+
+class SpeedSearchAgent implements SpeedSearchAgentType {
+  private config: Config;
+  private strParser = new StringOutputParser();
+  private searchQuery?: string;
+  private searxngUrl?: string;
+
+  constructor(config: Config) {
+    this.config = config;
+  }
+
+  /**
+   * Emit a progress event with the given percentage and message
+   */
+  private emitProgress(
+    emitter: eventEmitter,
+    percentage: number,
+    message: string,
+    subMessage?: string,
+  ) {
+    const progressData: any = {
+      message,
+      current: percentage,
+      total: 100,
+    };
+
+    // Add subMessage if provided
+    if (subMessage) {
+      progressData.subMessage = subMessage;
+    }
+
+    emitter.emit(
+      'progress',
+      JSON.stringify({
+        type: 'progress',
+        data: progressData,
+      }),
+    );
+  }
+
+  private async createSearchRetrieverChain(
+    llm: BaseChatModel,
+    systemInstructions: string,
+    emitter: eventEmitter,
+    signal: AbortSignal,
+  ) {
+    // TODO: Don't we want to set this back to default once search is done?
+    (llm as unknown as ChatOpenAI).temperature = 0;
+
+    this.emitProgress(emitter, 10, `Building search query`);
+
+    return RunnableSequence.from([
+      PromptTemplate.fromTemplate(this.config.queryGeneratorPrompt),
+      llm,
+      this.strParser,
+      RunnableLambda.from(async (input: string) => {
+        try {
+          //console.log(`LLM response for initial web search:"${input}"`);
+          const linksOutputParser = new LineListOutputParser({
+            key: 'links',
+          });
+
+          const questionOutputParser = new LineOutputParser({
+            key: 'answer',
+          });
+
+          const links = await linksOutputParser.parse(input);
+          let question = await questionOutputParser.parse(input);
+
+          //console.log('question', question);
+
+          if (question === 'not_needed') {
+            return { query: '', docs: [] };
+          }
+
+          if (links.length > 0) {
+            if (question.length === 0) {
+              question = 'summarize';
+            }
+
+            let docs: Document[] = [];
+
+            const linkDocs = await getDocumentsFromLinks({ links });
+
+            const docGroups: Document[] = [];
+
+            linkDocs.map((doc) => {
+              const URLDocExists = docGroups.find(
+                (d) =>
+                  d.metadata.url === doc.metadata.url &&
+                  d.metadata.totalDocs < 10,
+              );
+
+              if (!URLDocExists) {
+                docGroups.push({
+                  ...doc,
+                  metadata: {
+                    ...doc.metadata,
+                    totalDocs: 1,
+                  },
+                });
+              }
+
+              const docIndex = docGroups.findIndex(
+                (d) =>
+                  d.metadata.url === doc.metadata.url &&
+                  d.metadata.totalDocs < 10,
+              );
+
+              if (docIndex !== -1) {
+                docGroups[docIndex].pageContent =
+                  docGroups[docIndex].pageContent + `\n\n` + doc.pageContent;
+                docGroups[docIndex].metadata.totalDocs += 1;
+              }
+            });
+
+            this.emitProgress(emitter, 20, `Summarizing content`);
+
+            await Promise.all(
+              docGroups.map(async (doc) => {
+                const systemPrompt = systemInstructions
+                  ? `${systemInstructions}\n\n`
+                  : '';
+
+                const res = await llm.invoke(
+                  `${systemPrompt}You are a web search summarizer, tasked with summarizing a piece of text retrieved from a web search. Your job is to summarize the 
+            text into a detailed, 2-4 paragraph explanation that captures the main ideas and provides a comprehensive answer to the query.
+            If the query is \"summarize\", you should provide a detailed summary of the text. If the query is a specific question, you should answer it in the summary.
+            
+            - **Journalistic tone**: The summary should sound professional and journalistic, not too casual or vague.
+            - **Thorough and detailed**: Ensure that every key point from the text is captured and that the summary directly answers the query.
+            - **Not too lengthy, but detailed**: The summary should be informative but not excessively long. Focus on providing detailed information in a concise format.
+
+            The text will be shared inside the \`text\` XML tag, and the query inside the \`query\` XML tag.
+
+            <example>
+            1. \`<text>
+            Docker is a set of platform-as-a-service products that use OS-level virtualization to deliver software in packages called containers. 
+            It was first released in 2013 and is developed by Docker, Inc. Docker is designed to make it easier to create, deploy, and run applications 
+            by using containers.
+            </text>
+
+            <query>
+            What is Docker and how does it work?
+            </query>
+
+            Response:
+            Docker is a revolutionary platform-as-a-service product developed by Docker, Inc., that uses container technology to make application 
+            deployment more efficient. It allows developers to package their software with all necessary dependencies, making it easier to run in 
+            any environment. Released in 2013, Docker has transformed the way applications are built, deployed, and managed.
+            \`
+            2. \`<text>
+            The theory of relativity, or simply relativity, encompasses two interrelated theories of Albert Einstein: special relativity and general
+            relativity. However, the word "relativity" is sometimes used in reference to Galilean invariance. The term "theory of relativity" was based
+            on the expression "relative theory" used by Max Planck in 1906. The theory of relativity usually encompasses two interrelated theories by
+            Albert Einstein: special relativity and general relativity. Special relativity applies to all physical phenomena in the absence of gravity.
+            General relativity explains the law of gravitation and its relation to other forces of nature. It applies to the cosmological and astrophysical
+            realm, including astronomy.
+            </text>
+
+            <query>
+            summarize
+            </query>
+
+            Response:
+            The theory of relativity, developed by Albert Einstein, encompasses two main theories: special relativity and general relativity. Special
+            relativity applies to all physical phenomena in the absence of gravity, while general relativity explains the law of gravitation and its
+            relation to other forces of nature. The theory of relativity is based on the concept of "relative theory," as introduced by Max Planck in
+            1906. It is a fundamental theory in physics that has revolutionized our understanding of the universe.
+            \`
+            </example>
+
+            Everything below is the actual data you will be working with. Good luck!
+
+            <query>
+            ${question}
+            </query>
+
+            <text>
+            ${doc.pageContent}
+            </text>
+
+            Make sure to answer the query in the summary.
+          `,
+                  { signal },
+                );
+
+                const document = new Document({
+                  pageContent: res.content as string,
+                  metadata: {
+                    title: doc.metadata.title,
+                    url: doc.metadata.url,
+                  },
+                });
+
+                docs.push(document);
+              }),
+            );
+
+            return { query: question, docs: docs };
+          } else {
+            if (this.config.additionalSearchCriteria) {
+              question = `${question} ${this.config.additionalSearchCriteria}`;
+            }
+            this.emitProgress(
+              emitter,
+              20,
+              `Searching the web`,
+              `Search Query: ${question}`,
+            );
+
+            const searxngResult = await searchSearxng(question, {
+              language: 'en',
+              engines: this.config.activeEngines,
+            });
+
+            // Store the SearXNG URL for later use in emitting to the client
+            this.searxngUrl = searxngResult.searchUrl;
+
+            const documents = searxngResult.results.map(
+              (result) =>
+                new Document({
+                  pageContent:
+                    result.content ||
+                    (this.config.activeEngines.includes('youtube')
+                      ? result.title
+                      : '') /* Todo: Implement transcript grabbing using Youtubei (source: https://www.npmjs.com/package/youtubei) */,
+                  metadata: {
+                    title: result.title,
+                    url: result.url,
+                    ...(result.img_src && { img_src: result.img_src }),
+                  },
+                }),
+            );
+
+            return { query: question, docs: documents, searchQuery: question };
+          }
+        } catch (error) {
+          console.error('Error in search retriever chain:', error);
+          emitter.emit('error', JSON.stringify({ data: error }));
+          throw error;
+        }
+      }),
+    ]);
+  }
+
+  private async createAnsweringChain(
+    llm: BaseChatModel,
+    embeddings: Embeddings,
+    systemInstructions: string,
+    signal: AbortSignal,
+    emitter: eventEmitter,
+    personaInstructions?: string,
+  ) {
+    return RunnableSequence.from([
+      RunnableMap.from({
+        systemInstructions: () => systemInstructions,
+        query: (input: BasicChainInput) => input.query,
+        chat_history: (input: BasicChainInput) => input.chat_history,
+        date: () => formatDateForLLM(),
+        personaInstructions: () => personaInstructions || '',
+        context: RunnableLambda.from(
+          async (
+            input: BasicChainInput,
+            options?: { signal?: AbortSignal },
+          ) => {
+            // Check if the request was aborted
+            if (options?.signal?.aborted || signal?.aborted) {
+              console.log('Request cancelled by user');
+              throw new Error('Request cancelled by user');
+            }
+
+            const processedHistory = formatChatHistoryAsString(
+              input.chat_history,
+            );
+
+            let docs: Document[] | null = null;
+            let query = input.query;
+
+            if (this.config.searchWeb) {
+              const searchRetrieverChain =
+                await this.createSearchRetrieverChain(
+                  llm,
+                  systemInstructions,
+                  emitter,
+                  signal,
+                );
+              var date = formatDateForLLM();
+
+              const searchRetrieverResult = await searchRetrieverChain.invoke(
+                {
+                  chat_history: processedHistory,
+                  query,
+                  date,
+                  systemInstructions,
+                },
+                { signal: options?.signal },
+              );
+
+              query = searchRetrieverResult.query;
+              docs = searchRetrieverResult.docs;
+
+              // Store the search query in the context for emitting to the client
+              if (searchRetrieverResult.searchQuery) {
+                this.searchQuery = searchRetrieverResult.searchQuery;
+              }
+            }
+
+            const sortedDocs = await this.rerankDocsForSpeed(
+              query,
+              docs ?? [],
+              embeddings,
+              emitter,
+              signal,
+            );
+
+            if (options?.signal?.aborted || signal?.aborted) {
+              console.log('Request cancelled by user');
+              throw new Error('Request cancelled by user');
+            }
+
+            this.emitProgress(emitter, 100, `Done`);
+            return sortedDocs;
+          },
+        )
+          .withConfig({
+            runName: 'FinalSourceRetriever',
+          })
+          .pipe(this.processDocs),
+      }),
+      ChatPromptTemplate.fromMessages([
+        ['system', this.config.responsePrompt],
+        new MessagesPlaceholder('chat_history'),
+        ['user', '{query}'],
+      ]),
+      llm,
+      this.strParser,
+    ]).withConfig({
+      runName: 'FinalResponseGenerator',
+    });
+  }
+
+  /**
+   * Speed-optimized document reranking with simplified logic for web results only
+   */
+  private async rerankDocsForSpeed(
+    query: string,
+    docs: Document[],
+    embeddings: Embeddings,
+    emitter: eventEmitter,
+    signal: AbortSignal,
+  ): Promise<Document[]> {
+    try {
+      if (docs.length === 0) {
+        return docs;
+      }
+
+      if (query.toLocaleLowerCase() === 'summarize') {
+        return docs.slice(0, 15);
+      }
+
+      // Filter out documents with no content
+      let docsWithContent = docs.filter(
+        (doc) => doc.pageContent && doc.pageContent.length > 0,
+      );
+
+      // Speed mode logic - simply return first 15 documents with content
+      // No similarity ranking to prioritize speed
+      this.emitProgress(
+        emitter,
+        50,
+        `Ranking sources`,
+        this.searchQuery ? `Search Query: ${this.searchQuery}` : undefined,
+      );
+
+      return docsWithContent.slice(0, 15);
+    } catch (error) {
+      console.error('Error in rerankDocsForSpeed:', error);
+      emitter.emit('error', JSON.stringify({ data: error }));
+    }
+    return [];
+  }
+
+  private processDocs(docs: Document[]) {
+    const fullDocs = docs
+      .map(
+        (_, index) =>
+          `<${index + 1}>\n
+<title>${docs[index].metadata.title}</title>\n
+${docs[index].metadata?.url.toLowerCase().includes('file') ? '' : '\n<url>' + docs[index].metadata.url + '</url>\n'}
+<content>\n${docs[index].pageContent}\n</content>\n
+</${index + 1}>\n`,
+      )
+      .join('\n');
+    console.log('Processed docs:', fullDocs);
+    return fullDocs;
+  }
+
+  private async handleStream(
+    stream: AsyncGenerator<StreamEvent, any, any>,
+    emitter: eventEmitter,
+    llm: BaseChatModel,
+    signal: AbortSignal,
+  ) {
+    if (signal.aborted) {
+      return;
+    }
+
+    for await (const event of stream) {
+      if (signal.aborted) {
+        return;
+      }
+
+      if (
+        event.event === 'on_chain_end' &&
+        event.name === 'FinalSourceRetriever'
+      ) {
+        const sourcesData = event.data.output;
+        if (this.searchQuery) {
+          emitter.emit(
+            'data',
+            JSON.stringify({
+              type: 'sources',
+              data: sourcesData,
+              searchQuery: this.searchQuery,
+              searchUrl: this.searxngUrl,
+            }),
+          );
+        } else {
+          emitter.emit(
+            'data',
+            JSON.stringify({ type: 'sources', data: sourcesData }),
+          );
+        }
+      }
+      if (
+        event.event === 'on_chain_stream' &&
+        event.name === 'FinalResponseGenerator'
+      ) {
+        emitter.emit(
+          'data',
+          JSON.stringify({ type: 'response', data: event.data.chunk }),
+        );
+      }
+      if (
+        event.event === 'on_chain_end' &&
+        event.name === 'FinalResponseGenerator'
+      ) {
+        const modelName = getModelName(llm);
+
+        // Send model info before ending
+        emitter.emit(
+          'stats',
+          JSON.stringify({
+            type: 'modelStats',
+            data: {
+              modelName,
+            },
+          }),
+        );
+
+        emitter.emit('end');
+      }
+    }
+  }
+
+  async searchAndAnswer(
+    message: string,
+    history: BaseMessage[],
+    llm: BaseChatModel,
+    embeddings: Embeddings,
+    systemInstructions: string,
+    signal: AbortSignal,
+    personaInstructions?: string,
+    focusMode?: string,
+  ) {
+    const emitter = new eventEmitter();
+
+    const answeringChain = await this.createAnsweringChain(
+      llm,
+      embeddings,
+      systemInstructions,
+      signal,
+      emitter,
+      personaInstructions,
+    );
+
+    const stream = answeringChain.streamEvents(
+      {
+        chat_history: history,
+        query: message,
+      },
+      {
+        version: 'v1',
+        // Pass the abort signal to the LLM streaming chain
+        signal,
+      },
+    );
+
+    this.handleStream(stream, emitter, llm, signal);
+
+    return emitter;
+  }
+}
+
+export default SpeedSearchAgent;
--- a/src/lib/searxng.ts
+++ b/src/lib/searxng.ts
@ -19,6 +19,12 @@ interface SearxngSearchResult {
  iframe_src?: string;
 }

+interface SearxngResponse {
+  results: SearxngSearchResult[];
+  suggestions: string[];
+  searchUrl: string;
+}
+
 export const searchSearxng = async (
  query: string,
  opts?: SearxngSearchOptions,
@ -44,5 +50,16 @@ export const searchSearxng = async (
  const results: SearxngSearchResult[] = res.data.results;
  const suggestions: string[] = res.data.suggestions;

-  return { results, suggestions };
+  // Create a URL for viewing the search results in the SearXNG web interface
+  const searchUrl = new URL(searxngURL);
+  searchUrl.pathname = '/search';
+  searchUrl.searchParams.append('q', query);
+  if (opts?.engines?.length) {
+    searchUrl.searchParams.append('engines', opts.engines.join(','));
+  }
+  if (opts?.language) {
+    searchUrl.searchParams.append('language', opts.language);
+  }
+
+  return { results, suggestions, searchUrl: searchUrl.toString() };
 };
--- a/src/lib/utils.ts
+++ b/src/lib/utils.ts
@ -25,3 +25,16 @@ export const formatTimeDifference = (
  else
    return `${Math.floor(diffInSeconds / 31536000)} year${Math.floor(diffInSeconds / 31536000) !== 1 ? 's' : ''}`;
 };
+
+/**
+ * Format a date for LLM prompts to only include month, day, and year
+ * @param date - The date to format (defaults to current date)
+ * @returns Formatted date string in "Month DD, YYYY" format
+ */
+export const formatDateForLLM = (date: Date = new Date()): string => {
+  return date.toLocaleDateString('en-US', {
+    year: 'numeric',
+    month: 'long',
+    day: 'numeric',
+  });
+};
--- a/src/lib/utils/analyzePreviewContent.ts
+++ b/src/lib/utils/analyzePreviewContent.ts
@ -0,0 +1,158 @@
+import { BaseChatModel } from '@langchain/core/language_models/chat_models';
+import { BaseMessage } from '@langchain/core/messages';
+import { z } from 'zod';
+import { formatDateForLLM } from '../utils';
+import { ChatOpenAI, OpenAIClient } from '@langchain/openai';
+import { removeThinkingBlocks } from './contentUtils';
+import { withStructuredOutput } from './structuredOutput';
+
+export type PreviewAnalysisResult = {
+  isSufficient: boolean;
+  reason?: string;
+};
+
+export type PreviewContent = {
+  title: string;
+  snippet: string;
+  url: string;
+};
+
+// Zod schema for structured preview analysis output
+const PreviewAnalysisSchema = z.object({
+  isSufficient: z
+    .boolean()
+    .describe(
+      'Whether the preview content is sufficient to answer the task query',
+    ),
+  reason: z
+    .string()
+    .nullable()
+    .describe(
+      'Specific reason why full content analysis is required (only if isSufficient is false)',
+    ),
+});
+
+export const analyzePreviewContent = async (
+  previewContents: PreviewContent[],
+  query: string,
+  taskQuery: string,
+  chatHistory: BaseMessage[],
+  llm: BaseChatModel,
+  systemInstructions: string,
+  signal: AbortSignal,
+): Promise<PreviewAnalysisResult> => {
+  try {
+    console.log(`Analyzing preview content for query: "${query}"`);
+    console.log(
+      `Preview content being analyzed:`,
+      previewContents.map((content) => ({
+        title: content.title,
+        snippet: content.snippet.substring(0, 100) + '...',
+        url: content.url,
+      })),
+    );
+
+    // Format preview content for analysis
+    const formattedPreviewContent = previewContents
+      .map(
+        (content, index) =>
+          `Source ${index + 1}:
+Title: ${content.title}
+Snippet: ${content.snippet}
+---`,
+      )
+      .join('\n\n');
+
+    // Format chat history for context
+    const formattedChatHistory = chatHistory
+      .slice(-10) // Only include last 10 messages for context
+      .map(
+        (message) =>
+          `${message.getType()}: ${removeThinkingBlocks(message.content.toString())}`,
+      )
+      .join('\n');
+
+    const systemPrompt = systemInstructions ? `${systemInstructions}\n\n` : '';
+
+    console.log(`Invoking LLM for preview content analysis`);
+
+    // Create structured LLM with Zod schema
+    const structuredLLM = withStructuredOutput(llm, PreviewAnalysisSchema, {
+      name: 'analyze_preview_content',
+    });
+
+    const analysisResult = await structuredLLM.invoke(
+      `You are a preview content analyzer, tasked with determining if search result snippets contain sufficient information to answer the Task Query.
+
+# Instructions
+- Analyze the provided search result previews (titles + snippets), and chat history context to determine if they collectively contain enough information to provide a complete and accurate answer to the Task Query
+- If the preview content can provide a complete answer to the Task Query, consider it sufficient
+- If the preview content lacks important details, requires deeper analysis, or cannot fully answer the Task Query, consider it insufficient
+- Be specific in your reasoning when the content is not sufficient
+- The original query is provided for additional context, only use it for clarification of overall expectations and intent. You do **not** need to answer the original query directly or completely
+
+# System Instructions
+${systemPrompt}
+
+# Response Format
+Respond with a JSON object that matches this structure:
+{
+  "isSufficient": boolean,
+  "reason": "string"
+}
+
+Your response should contain only the JSON object, no additional text or formatting.
+
+# Information Context:
+Today's date is ${formatDateForLLM(new Date())}
+
+# Chat History Context:
+${formattedChatHistory ? formattedChatHistory : 'No previous conversation context'}
+
+# User Query:
+${query}
+
+# Task Query (what to answer):
+${taskQuery}
+
+# Search Result Previews to Analyze:
+${formattedPreviewContent}
+`,
+      { signal },
+    );
+
+    if (!analysisResult) {
+      console.error('No analysis result returned from LLM');
+      return {
+        isSufficient: false,
+        reason:
+          'No analysis response returned from LLM - falling back to full content processing',
+      };
+    }
+
+    console.log(`LLM analysis result:`, analysisResult);
+
+    if (analysisResult.isSufficient) {
+      console.log(
+        'Preview content determined to be sufficient for answering the query',
+      );
+      return { isSufficient: true };
+    } else {
+      console.log(
+        `Preview content determined to be insufficient. Reason: ${analysisResult.reason}`,
+      );
+      return {
+        isSufficient: false,
+        reason:
+          analysisResult.reason ||
+          'Preview content insufficient for complete answer',
+      };
+    }
+  } catch (error) {
+    console.error('Error analyzing preview content:', error);
+    return {
+      isSufficient: false,
+      reason: `Error during preview analysis: ${error instanceof Error ? error.message : 'Unknown error'} - falling back to full content processing`,
+    };
+  }
+};
--- a/src/lib/utils/contentUtils.ts
+++ b/src/lib/utils/contentUtils.ts
@ -0,0 +1,48 @@
+import {
+  BaseMessage,
+  AIMessage,
+  HumanMessage,
+  SystemMessage,
+} from '@langchain/core/messages';
+
+/**
+ * Removes all content within <think>...</think> blocks
+ * @param text The input text containing thinking blocks
+ * @returns The text with all thinking blocks removed
+ */
+export const removeThinkingBlocks = (text: string): string => {
+  // Use regex to identify and remove all <think>...</think> blocks
+  // Using the 's' flag to make dot match newlines
+  return text.replace(/<think>[\s\S]*?<\/think>/g, '').trim();
+};
+
+/**
+ * Removes thinking blocks from the content of an array of BaseMessage objects
+ * @param messages Array of BaseMessage objects
+ * @returns New array with thinking blocks removed from each message's content
+ */
+export const removeThinkingBlocksFromMessages = (
+  messages: BaseMessage[],
+): BaseMessage[] => {
+  return messages.map((message) => {
+    // Only process string content, leave complex content as-is
+    if (typeof message.content !== 'string') {
+      return message;
+    }
+
+    const cleanedContent = removeThinkingBlocks(message.content);
+
+    // Create new instance of the same message type with cleaned content
+    if (message instanceof AIMessage) {
+      return new AIMessage(cleanedContent);
+    } else if (message instanceof HumanMessage) {
+      return new HumanMessage(cleanedContent);
+    } else if (message instanceof SystemMessage) {
+      return new SystemMessage(cleanedContent);
+    } else {
+      // For any other message types, return the original message unchanged
+      // This is a safe fallback for custom message types
+      return message;
+    }
+  });
+};
--- a/src/lib/utils/documents.ts
+++ b/src/lib/utils/documents.ts
@ -1,8 +1,15 @@
-import axios from 'axios';
-import { htmlToText } from 'html-to-text';
-import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';
+import { CheerioWebBaseLoader } from '@langchain/community/document_loaders/web/cheerio';
+import { PlaywrightWebBaseLoader } from '@langchain/community/document_loaders/web/playwright';
 import { Document } from '@langchain/core/documents';
+import { Embeddings } from '@langchain/core/embeddings';
+import { Readability } from '@mozilla/readability';
+import axios from 'axios';
+import { JSDOM } from 'jsdom';
+import { RecursiveCharacterTextSplitter } from 'langchain/text_splitter';
+import fetch from 'node-fetch';
 import pdfParse from 'pdf-parse';
+import type { Browser, Page } from 'playwright';
+import computeSimilarity from './computeSimilarity';

 export const getDocumentsFromLinks = async ({ links }: { links: string[] }) => {
  const splitter = new RecursiveCharacterTextSplitter();
@ -17,13 +24,16 @@ export const getDocumentsFromLinks = async ({ links }: { links: string[] }) => {
          : `https://${link}`;

      try {
+        // First, check if it's a PDF
+        const headRes = await axios.head(link);
+        const isPdf = headRes.headers['content-type'] === 'application/pdf';
+
+        if (isPdf) {
+          // Handle PDF files
          const res = await axios.get(link, {
            responseType: 'arraybuffer',
          });

-        const isPdf = res.headers['content-type'] === 'application/pdf';
-
-        if (isPdf) {
          const pdfText = await pdfParse(res.data);
          const parsedText = pdfText.text
            .replace(/(\r\n|\n|\r)/gm, ' ')
@ -47,36 +57,29 @@ export const getDocumentsFromLinks = async ({ links }: { links: string[] }) => {
          return;
        }

-        const parsedText = htmlToText(res.data.toString('utf8'), {
-          selectors: [
-            {
-              selector: 'a',
-              options: {
-                ignoreHref: true,
-              },
-            },
-          ],
-        })
-          .replace(/(\r\n|\n|\r)/gm, ' ')
-          .replace(/\s+/g, ' ')
-          .trim();
+        // Handle web pages using CheerioWebBaseLoader
+        const loader = new CheerioWebBaseLoader(link, {
+          selector: 'body',
+        });

-        const splittedText = await splitter.splitText(parsedText);
-        const title = res.data
-          .toString('utf8')
-          .match(/<title.*>(.*?)<\/title>/)?.[1];
+        const webDocs = await loader.load();
+
+        if (webDocs && webDocs.length > 0) {
+          const webDoc = webDocs[0];
+          const splittedText = await splitter.splitText(webDoc.pageContent);

          const linkDocs = splittedText.map((text) => {
            return new Document({
              pageContent: text,
              metadata: {
-              title: title || link,
+                title: webDoc.metadata.title || link,
                url: link,
              },
            });
          });

          docs.push(...linkDocs);
+        }
      } catch (err) {
        console.error(
          'An error occurred while getting documents from links: ',
@ -97,3 +100,322 @@ export const getDocumentsFromLinks = async ({ links }: { links: string[] }) => {

  return docs;
 };
+
+/**
+ * Fetches web content from a given URL using LangChain's PlaywrightWebBaseLoader.
+ * Parses it using Readability for better content extraction.
+ * Returns a Document object containing the parsed text and metadata.
+ *
+ * @param url - The URL to fetch content from.
+ * @param getHtml - Whether to include the HTML content in the metadata.
+ * @returns A Promise that resolves to a Document object or null if parsing fails.
+ */
+export const getWebContent = async (
+  url: string,
+  getHtml: boolean = false,
+): Promise<Document | null> => {
+  try {
+    console.log(`Fetching content from URL: ${url}`);
+
+    const loader = new PlaywrightWebBaseLoader(url, {
+      launchOptions: {
+        headless: true,
+        timeout: 30000,
+      },
+      gotoOptions: {
+        waitUntil: 'domcontentloaded',
+        timeout: 10000,
+      },
+      async evaluate(page: Page, browser: Browser) {
+        // Wait for the content to load properly
+        await page.waitForLoadState('networkidle', { timeout: 10000 });
+
+        // Allow some time for dynamic content to load
+        await page.waitForTimeout(3000);
+
+        return await page.content();
+      },
+    });
+
+    const docs = await loader.load();
+
+    if (!docs || docs.length === 0) {
+      console.warn(`Failed to load content for URL: ${url}`);
+      return null;
+    }
+
+    const doc = docs[0];
+
+    const dom = new JSDOM(doc.pageContent, { url });
+    const reader = new Readability(dom.window.document, { charThreshold: 25 });
+    const article = reader.parse();
+
+    // Normalize the text content
+    const normalizedText =
+      article?.textContent
+        ?.split('\n')
+        .map((line: string) => line.trim())
+        .filter((line: string) => line.length > 0)
+        .join('\n') || '';
+
+    const returnDoc = new Document({
+      pageContent: normalizedText,
+      metadata: {
+        title: article?.title || doc.metadata.title || '',
+        url: url,
+        html: getHtml ? article?.content : undefined,
+      },
+    });
+
+    console.log(
+      `Got content with LangChain Playwright, URL: ${url}, Text Length: ${returnDoc.pageContent.length}`,
+    );
+
+    return returnDoc;
+  } catch (error) {
+    console.error(`Error fetching/parsing URL ${url}:`, error);
+
+    // Fallback to CheerioWebBaseLoader for simpler content extraction
+    try {
+      console.log(`Fallback to Cheerio for URL: ${url}`);
+      const cheerioLoader = new CheerioWebBaseLoader(url);
+      const docs = await cheerioLoader.load();
+
+      if (docs && docs.length > 0) {
+        return docs[0];
+      }
+    } catch (fallbackError) {
+      console.error(
+        `Cheerio fallback also failed for URL ${url}:`,
+        fallbackError,
+      );
+    }
+
+    return null;
+  }
+};
+
+/**
+ * Fetches web content from a given URL using CheerioWebBaseLoader for faster, lighter extraction.
+ * Returns a Document object containing the parsed text and metadata.
+ *
+ * @param {string} url - The URL to fetch content from.
+ * @param {boolean} getHtml - Whether to include the HTML content in the metadata.
+ * @returns {Promise<Document | null>} A Promise that resolves to a Document object or null if parsing fails.
+ */
+export const getWebContentLite = async (
+  url: string,
+  getHtml: boolean = false,
+): Promise<Document | null> => {
+  try {
+    console.log(`Fetching content (lite) from URL: ${url}`);
+
+    const loader = new CheerioWebBaseLoader(url);
+
+    const docs = await loader.load();
+
+    if (!docs || docs.length === 0) {
+      console.warn(`Failed to load content for URL: ${url}`);
+      return null;
+    }
+
+    const doc = docs[0];
+
+    // Try to use Readability for better content extraction if possible
+    if (getHtml) {
+      try {
+        const response = await fetch(url, { timeout: 5000 });
+        const html = await response.text();
+        const dom = new JSDOM(html, { url });
+        const originalTitle = dom.window.document.title;
+        const reader = new Readability(dom.window.document, {
+          charThreshold: 25,
+        });
+        const article = reader.parse();
+
+        if (article) {
+          const normalizedText =
+            article.textContent
+              ?.split('\n')
+              .map((line) => line.trim())
+              .filter((line) => line.length > 0)
+              .join('\n') || '';
+
+          return new Document({
+            pageContent: normalizedText,
+            metadata: {
+              html: article.content,
+              title: article.title || originalTitle,
+              url: url,
+            },
+          });
+        }
+      } catch (readabilityError) {
+        console.warn(
+          `Readability parsing failed for ${url}, using Cheerio fallback`,
+        );
+      }
+    }
+
+    // Normalize the text content from Cheerio
+    const normalizedText = doc.pageContent
+      .split('\n')
+      .map((line: string) => line.trim())
+      .filter((line: string) => line.length > 0)
+      .join('\n');
+
+    return new Document({
+      pageContent: normalizedText,
+      metadata: {
+        title: doc.metadata.title || 'Web Page',
+        url: url,
+        html: getHtml ? doc.pageContent : undefined,
+      },
+    });
+  } catch (error) {
+    console.error(`Error fetching/parsing URL ${url}:`, error);
+    return null;
+  }
+};
+
+/**
+ * Fetches web content from a given URL using LangChain's PlaywrightWebBaseLoader.
+ * Parses it using Readability for better content extraction.
+ * Returns a Document object containing relevant snippets of text using ranked text splitting.
+ * Text is split into chunks of approximately 800 characters, with 100 characters overlap.
+ *
+ * @param url - The URL to fetch content from.
+ * @param rankAgainstVector - The vector to rank the content against for relevance.
+ * @param embeddings - The embeddings model to use for ranking the content.
+ * @returns A Promise that resolves to a Document object or null if parsing fails.
+ */
+export const getRankedWebContentSnippets = async (
+  url: string,
+  rankAgainstVector: number[],
+  embeddings: Embeddings,
+): Promise<Document | null> => {
+  try {
+    console.log(`Fetching ranked content snippets from URL: ${url}`);
+
+    const loader = new PlaywrightWebBaseLoader(url, {
+      launchOptions: {
+        headless: true,
+        timeout: 30000,
+      },
+      gotoOptions: {
+        waitUntil: 'domcontentloaded',
+        timeout: 10000,
+      },
+      async evaluate(page: Page, browser: Browser) {
+        // Wait for the content to load properly
+        await page.waitForLoadState('networkidle', { timeout: 10000 });
+
+        // Allow some time for dynamic content to load
+        await page.waitForTimeout(3000);
+
+        return await page.content();
+      },
+    });
+
+    const docs = await loader.load();
+
+    if (!docs || docs.length === 0) {
+      console.warn(`Failed to load content for URL: ${url}`);
+      return null;
+    }
+
+    const doc = docs[0];
+
+    const dom = new JSDOM(doc.pageContent, { url });
+    const reader = new Readability(dom.window.document, {
+      charThreshold: 25,
+    });
+    const article = reader.parse();
+
+    // Split text into chunks with specified parameters
+    const splitter = RecursiveCharacterTextSplitter.fromLanguage('html', {
+      chunkSize: 800,
+      chunkOverlap: 100,
+    });
+
+    const textChunks = await splitter.splitText(article?.content || '');
+    if (!textChunks || textChunks.length === 0) {
+      console.warn(`No text chunks found for URL: ${url}`);
+      return null;
+    }
+
+    const similarity = await Promise.all(
+      textChunks.map(async (chunk, i) => {
+        const sim = computeSimilarity(
+          rankAgainstVector,
+          (await embeddings.embedDocuments([chunk]))[0],
+        );
+        return {
+          index: i,
+          similarity: sim,
+        };
+      }),
+    );
+
+    let rankedChunks = similarity
+      .sort((a, b) => b.similarity - a.similarity)
+      .map((sim) => textChunks[sim.index])
+      .slice(0, 5);
+
+    // Combine chunks into a single document with the most relevant content
+    const combinedContent = rankedChunks.join('\n\n');
+
+    const returnDoc = new Document({
+      pageContent: combinedContent,
+      metadata: {
+        title: article?.title || doc.metadata.title || '',
+        url: url,
+        chunks: rankedChunks.length,
+      },
+    });
+
+    console.log(
+      `Got ranked content snippets, URL: ${url}, Chunks: ${rankedChunks.length}, Total Length: ${returnDoc.pageContent.length}`,
+    );
+
+    return returnDoc;
+  } catch (error) {
+    console.error(`Error fetching/parsing URL ${url}:`, error);
+
+    // Fallback to CheerioWebBaseLoader for simpler content extraction
+    // try {
+    //   console.log(`Fallback to Cheerio for URL: ${url}`);
+    //   const cheerioLoader = new CheerioWebBaseLoader(url);
+    //   const docs = await cheerioLoader.load();
+
+    //   if (docs && docs.length > 0) {
+    //     const doc = docs[0];
+
+    //     // Apply the same splitting logic to fallback content
+    //     const splitter = new RecursiveCharacterTextSplitter({
+    //       chunkSize: 800,
+    //       chunkOverlap: 100,
+    //     });
+
+    //     const textChunks = await splitter.splitText(doc.pageContent);
+    //     const combinedContent = textChunks.join('\n\n');
+
+    //     return new Document({
+    //       pageContent: combinedContent,
+    //       metadata: {
+    //         title: doc.metadata.title || '',
+    //         url: url,
+    //         chunks: textChunks.length,
+    //       },
+    //     });
+    //   }
+    // } catch (fallbackError) {
+    //   console.error(
+    //     `Cheerio fallback also failed for URL ${url}:`,
+    //     fallbackError,
+    //   );
+    // }
+
+    return null;
+  }
+};
--- a/Show more
+++ b/Show more