Commit graph

55 commits

Author SHA1 Message Date
hlohaus
4399b432c4 Refactor OpenaiChat authentication flow; replace get_nodriver with async context manager and improve error handling
Update backend_anon_url in har_file.py for correct endpoint
Add async context manager for get_nodriver_session in requests module
Fix start-browser.sh to remove stale cookie file before launching Chrome
2025-10-02 02:08:20 +02:00
hlohaus
4e5c26fd60 Remove unused import and update sse_stream function signature for consistency 2025-09-21 11:40:54 +02:00
hlohaus
4f44c149ba Refactor BrowserConfig class: move it from requests to cookies module and update its initialization. 2025-09-06 21:22:15 +02:00
hlohaus
bca45481ea Refactor SharedTokenManager to remove async from reloadCredentialsFromFile and update its invocation. Set default host in BrowserConfig to 127.0.0.1. 2025-09-06 20:22:15 +02:00
hlohaus
933fff985b Enhance PuterJS provider to track live model instances and update BrowserConfig for automation tool settings 2025-09-06 16:28:17 +02:00
hlohaus
0b75e68b51 Update styling 2025-08-25 00:19:42 +02:00
hlohaus
05f0f55711 refactor: update providers list, env vars, and minor fixes
- **example.env**:
  - Added `G4F_API_KEY` and `G4F_PROXY` variables
  - Removed Azure-related API keys and routes block
  - Added `OLLAMA_API_KEY`, `NVIDIA_API_KEY`, and `PUTER_API_KEY`

- **g4f/Provider/EasyChat.py**:
  - Replaced `page.js_dumps` with `page.evaluate` for retrieving `guestId`

- **g4f/Provider/deprecated/LegacyLMArena.py**:
  - Changed `working = True` to `working = False`

- **g4f/Provider/deprecated/har/__init__.py**:
  - Changed `working = True` to `working = False`

- **g4f/providers/any_provider.py**:
  - Updated imports, adding `Custom`, `PollinationsImage`, `OpenaiAccount`, and reordering providers
  - Renamed `PROVIERS_LIST_2` → `PROVIDERS_LIST_2` and `PROVIERS_LIST_3` → `PROVIDERS_LIST_3`
  - Removed `PROVIERS_LIST_1` and replaced its usage with `Provider.__providers__`
  - Changed `cls.audio_models` initialization from `{}` to `[]`
  - Adjusted handling of model mapping with `clean_name` for consistency
  - GeminiPro check updated to `if provider == GeminiPro`
  - Changed final `cls.audio_models` assignment to `[ *cls.audio_models ]` instead of `list(cls.audio_models.keys())`
  - Adjusted provider fallback in `AnyProvider` to use `PROVIDERS_LIST_2 + PROVIDERS_LIST_3`

- **g4f/requests/__init__.py**:
  - Changed SSE parsing condition from `line.startswith(b"data: ")` to `line.startswith(b"data:")`
  - Updated slice from `line[6:]` to `line[5:]` when extracting `rest
2025-08-18 17:44:05 +02:00
hlohaus
4b1351a457 Add --no-sandbox to nodriver 2025-08-11 18:49:56 +02:00
hlohaus
116a01cd5e Update nodriver function 2025-08-11 17:24:41 +02:00
hlohaus
4132b0a494 feat: add image size support and update nodriver args handling
- **ApiAirforce.py**: Added `use_image_size = True` class attribute.
- **EasyChat.py**: Added `user_data_dir=None` argument when calling `get_args_from_nodriver`.
- **LMArenaBeta.py**:
  - Only set `cls.share_url` from `G4F_SHARE_URL` if `cls.share_url` is `None`.
  - Added `user_data_dir=None` argument when calling `get_args_from_nodriver`.
  - Modified media filtering to check `url` is a `str` before `startswith("https://")`.
- **OpenaiTemplate.py**:
  - Added `use_image_size = False` class attribute.
  - Modified image generation payload to optionally include `"size"` key when `use_image_size` is `True` and width/height present.
  - Refactored `read_response` to store `content` in a variable, strip on first chunk, and yield `ToolCalls` directly from `tool_calls` variable.
- **g4f/requests/__init__.py**:
  - Added `user_data_dir` parameter (default `"nodriver"`) to `get_args_from_nodriver`.
  - Passed `user_data_dir` to `get_nodriver` call.
2025-08-11 15:48:16 +02:00
hlohaus
b9dfe1a460 feat: add EasyChat and GLM providers, update HuggingFace and SSE parsing
- Added new `EasyChat` provider (`g4f/Provider/EasyChat.py`) with captcha handling, nodriver callback, and token caching
- Added new `GLM` provider (`g4f/Provider/GLM.py`) with model retrieval, auth token fetch, and SSE streaming support
- Updated `g4f/Provider/__init__.py` to import `EasyChat` and `GLM`
- Modified `LMArenaBeta` in `g4f/Provider/needs_auth/LMArenaBeta.py` to remove nodriver availability check and always use `get_args_from_nodriver` with callback
- Updated `HuggingFaceAPI` in `g4f/Provider/needs_auth/hf/HuggingFaceAPI.py` to use `default_model` from `models` instead of `default_llama_model` and removed commented `max_inputs_lenght` param
- Updated `HuggingFace` in `g4f/Provider/needs_auth/hf/__init__.py` to import `default_model` instead of `default_vision_model`, set `default_model` class attribute, and commented out HuggingFaceInference and image model handling logic
- Modified `OpenaiTemplate` in `g4f/Provider/template/OpenaiTemplate.py` to prefer `"name"` over `"id"` when populating `vision_models`, `models`, and `models_count`
- Enhanced `sse_stream` in `g4f/requests/__init__.py` to strip and skip empty `data:` lines, handle JSON decode errors, and raise `ValueError` on invalid JSON
2025-08-08 01:19:32 +02:00
hlohaus
499dcc0154 refactor: replace see_stream with sse_stream and update md2html output logic
- Replaced all imports and usages of `see_stream` with `sse_stream` across:
  - `g4f/Provider/Kimi.py`
  - `g4f/Provider/hf_space/BlackForestLabs_Flux1KontextDev.py`
  - `g4f/Provider/needs_auth/PuterJS.py`
  - `g4f/Provider/template/OpenaiTemplate.py`
  - `g4f/requests/__init__.py` (renamed function `see_stream` to `sse_stream`)
- Modified `g4f/Provider/needs_auth/GeminiPro.py`:
  - Updated `default_model` from `gemini-2.5-flash-preview-04-17` to `gemini-2.5-flash`
  - Removed `gemini-2.5-flash-preview-04-17` from `fallback_models`
- Updated `etc/tool/md2html.py`:
  - Added `re` import
  - Changed `process_single_file_with_output` to check if output file exists
  - If exists, uses regex to update `<title>` and `itemprop="text">` content instead of writing full template
  - If not, generates HTML using the template as before
2025-07-29 19:57:13 +02:00
hlohaus
ffa05f5c4d Fix streaming 2025-07-15 16:02:01 +02:00
hlohaus
c718011cb8 Add YouTube provider 2025-07-14 08:49:12 +02:00
hlohaus
62803546a6 Fix using nodriver 2025-07-12 11:45:21 +02:00
hlohaus
3c66fa114a feat: add 'transparent' image model and refactor HAR provider auth flow
- Added "transparent" to `image_models` in `PollinationsAI` and mapped it to "gptimage"
- Modified transparent flag handling in `_generate_image` call in `PollinationsAI`
- Removed unused `cls.get_models()` call from image generation method in `PollinationsAI`
- Replaced `AsyncGeneratorProvider` with `AsyncAuthedProvider` in `HarProvider`
- Implemented `on_auth_async` in `HarProvider` to support browser-based auth via `nodriver`
- Replaced `create_async_generator` with `create_authed` in `HarProvider` to support `AuthResult`
- Removed custom `headers` in HAR post requests; used `auth_result.get_dict()` for `StreamSession`
- Refactored `Video` provider to support optional search in `get_response`
- Added `search` parameter to `RequestConfig.get_response` and `Video.create_async_generator`
- Improved browser automation and element interaction logic in `Video` provider
- Extracted video request interception to collect URLs using `nodriver`
- Reduced video polling loop timeout from 600 to 300 iterations in `Video`
- Updated CLI `client.py` to fix handling of `conversation.conversation` assignment
- Fixed argparse config: removed `nargs='?'` and added `metavar` for `--conversation-file`
- Improved image metadata extraction in API and backend when Pillow is available
- Modified `ImageResponse.__str__` to output HTML anchor/image tags with dimensions if present
- Added support for returning `target_path` from `copy_media` if `return_target` is True
- Changed default image processing size in `process_image` from 800x400 to 400x400
- Disabled RGBA-to-RGB flattening in `process_image`
- Improved `get_args_from_nodriver` to ensure proper referer and cookie handling
- Added helper `get_target_paths_and_urls` in `Api` to extract image dimensions from disk paths
2025-07-11 01:46:32 +02:00
hlohaus
5e4b9d9866 fix: improve session handling, message formatting, and content saving
- Added `timeout` parameter support to `LMArenaBeta._create_async_generator` and passed it to `StreamSession`
- Ensured fallback to `default_model` in `LMArenaBeta` if `model` is not provided
- Modified `OpenaiChat._create_completion` to rebuild messages excluding assistant roles if conversation ID exists
- Corrected OpenaiChat `nodriver_auth` to await `browser.get` and replaced page access with reload
- Improved `save_content` in `client.py` with robust content extraction, null checks, and logging for missing content
- Removed premature `input_text.strip()` in `stream_response` and relocated it to `run_client_args`
- Simplified and centralized markdown filtering call in `save_content`
- Replaced raw `print` logging in `__init__.py` with `debug.log` for `nodriver` URL opening message
2025-06-29 21:59:22 +02:00
hlohaus
a30908ae0b Fix stop_browser, update secret validation 2025-06-27 14:10:30 +02:00
hlohaus
315495e199 Support inlineData in GeminiPro 2025-06-26 00:25:53 +02:00
hlohaus
a354a2e8f8 Update copy_image 2025-06-18 00:06:49 +02:00
hlohaus
93986d15f6 fix: resolve model duplication and improve provider handling
- Fixed duplicate model entries in Blackbox provider model_aliases
- Added meta-llama- to llama- name cleaning in Cloudflare provider
- Enhanced PollinationsAI provider with improved vision model detection
- Added reasoning support to PollinationsAI provider
- Fixed HuggingChat authentication to include headers and impersonate
- Removed unused max_inputs_length parameter from HuggingFaceAPI
- Renamed extra_data to extra_body for consistency across providers
- Added Puter provider with grouped model support
- Enhanced AnyProvider with grouped model display and better model organization
- Fixed model cleaning in AnyProvider to handle more model name variations
- Added api_key handling for HuggingFace providers in AnyProvider
- Added see_stream helper function to parse event streams
- Updated GUI server to handle JsonConversation properly
- Fixed aspect ratio handling in image generation functions
- Added ResponsesConfig and ClientResponse for new API endpoint
- Updated requirements to include markitdown
2025-05-16 00:18:12 +02:00
hlohaus
02384a616a Add set set_browser_executable_path 2025-04-20 22:16:01 +02:00
hlohaus
17976b93a7 ```
feat: add __call__ in field_serializer and update body wait logic

- In g4f/client/stubs.py, add a __call__ method to the field_serializer class that returns the first positional argument.
- In g4f/requests/__init__.py, replace the wait_for call with a while loop that repeatedly evaluates "document.querySelector('body:not(.no-js)')" and sleeps until the element is found.
```
2025-04-17 08:19:33 +02:00
hlohaus
06546649db feat: add LM Arena provider, async‑ify Copilot & surface follow‑up suggestions
* **Provider/Blackbox.py**
  * Raise `RateLimitError` when `"You have reached your request limit for the hour"` substring is detected
* **Provider/Copilot.py**
  * Convert class to `AsyncGeneratorProvider`; rename `create_completion` → `create_async_generator`
  * Swap `curl_cffi.requests.Session` for `AsyncSession`; reduce default timeout to **30 s**
  * Fully async websocket flow (`await session.ws_connect`, `await wss.send/recv/close`)
  * Emit new response types: `TitleGeneration`, `SourceLink`, aggregated `Sources`
  * Track request completion with `done` flag; collect citations in `sources` dict
* **Provider/DuckDuckGo.py**
  * Replace `duckduckgo_search.DDGS` with `duckai.DuckAI`
  * Change base class to `AbstractProvider`; drop nodriver‑based auth
* **Provider/PollinationsAI.py**
  * Re‑build text/audio model lists ensuring uniqueness; remove unused `extra_text_models`
  * Fix image seed logic (`i==1` for first retry); propagate streaming `error` field via `ResponseError`
* **Provider/hf_space**
  * **New file** `LMArenaProvider.py` implementing async queue/stream client
  * Register `LMArenaProvider` in `hf_space/__init__.py`; delete `G4F` import
* **Provider/needs_auth/CopilotAccount.py**
  * Inherit order changed to `Copilot, AsyncAuthedProvider`
  * Refactor token & cookie propagation; add `cookies_to_dict` helper
* **Provider/needs_auth/OpenaiChat.py**
  * Parse reasoning thoughts/summary; yield `Reasoning` responses
  * Tighten access‑token validation and nodriver JS evaluations (`return_by_value`)
  * Extend `Conversation` with `p` and `thoughts_summary`
* **providers/response.py**
  * Add `SourceLink` response class returning single formatted citation link
* **providers/base_provider.py**
  * Serialize `AuthResult` with custom `json.dump` to handle non‑serializable fields
  * Gracefully skip empty cache files when loading auth data
* **image/copy_images.py**
  * Ignore file extensions longer than 4 chars when inferring type
* **requests/__init__.py**
  * Use `return_by_value=True` for `navigator.userAgent` extraction
* **models.py**
  * Remove `G4F` from model provider lists; update `janus_pro_7b` best providers
* **GUI server/api.py**
  * Stream `SuggestedFollowups` to client (`"suggestions"` event)
* **GUI static assets**
  * **style.css**: bold chat title, add `.suggestions` styles, remove padding from `.chat-body`
  * **chat.v1.js**
    * Capture `suggestions` packets, render buttons, and send as quick replies
    * Re‑order finish‑reason logic; adjust token count placement and system‑prompt toggling
  * **chat-top-panel / footer** interactions updated accordingly
* **gui/client/static/js/chat.v1.js** & **css** further UI refinements (scroll handling, token counting, hide prompt toggle)
* Minor updates across multiple files to match new async interfaces and headers (`userAgent`, `raise_for_status`)
2025-04-17 01:21:58 +02:00
hlohaus
4ae536f1cc Fix issue with new nodriver version
Fix issue with stop parameter
2025-04-02 11:44:58 +02:00
hlohaus
fa17ce1bd6 Delete buckets with the conversation
Fix lightbox for uploaded images
Fix save media content
Fix TypeGPT and Cloudflare provider
2025-03-29 11:22:22 +01:00
hlohaus
a1871dafeb Improve background page
Improve share js functions
Fix photoswipe in UI
Support n parameter in PollinationsAI
2025-03-26 21:39:19 +01:00
hlohaus
b84f35f4e4 Improve model list in HuggingFace
WakeLook and disable count tokens for performance
Export and import settings in UI
Some styling improvments in UI
Fix curl_cffi updated bugs
2025-02-22 03:33:12 +01:00
hlohaus
b86bf0dcf7 Fix include in mocks.py 2025-02-03 21:24:37 +01:00
hlohaus
797b17833a Fix response type of reasoning in UI 2025-02-01 12:15:46 +01:00
hlohaus
89e096334d Support reasoning tokens by default
Add new default HuggingFace provider
Add format_image_prompt and get_last_user_message helper
Add stop_browser callable to get_nodriver function
Fix content type response in images route
2025-01-31 17:36:48 +01:00
hlohaus
30381f687e Add Janus_Pro_7B provider
Add proof of DeepSeekAPI provider
Add BackendApi provider from HuggingSpace
2025-01-30 00:35:51 +01:00
hlohaus
19bf5b7ef6 Fix upload files in demo mode
Updates for memory with mem0
Fix asyncio import in nodriver function
Add provider specific api endpoints
Support for open settings in UI at /chat/settings
2025-01-26 21:56:31 +01:00
hlohaus
ab04d1d894 Increase max token in HuggingfaceAPI
Restore browser instance on start up errors in nodriver
Restored instances can be used as usual or to stop the browser
Add demo modus to web ui for HuggingSpace
Add rate limit support to web ui. Simply install flask_limiter
Add home for demo with Access Token input and validation
Add stripped model list for demo
Add ignores for encoding error in web_search and file upload
2025-01-26 02:48:43 +01:00
hlohaus
3e1e505619 Add lock file for nodriver, add finally stop browser to all calls 2025-01-25 12:02:37 +01:00
hlohaus
09f8258fa7 Add MiniMax providers, add HailuoAI provider
Add OpenaiTemplate provider
Remove some webdriver providers
2025-01-25 03:56:37 +01:00
hlohaus
5651760ccf Add RequestLogin to HuggingChat 2025-01-24 00:42:35 +01:00
Heiner Lohaus
2e531d227c Fix invalid escape in requests module
Add none auth with OpenAI using nodriver
Fix missing 1 required positional argument: 'cls'
Update count tokens in GUI
Fix streaming example in requests guide
Remove ChatGptEs as default model
2025-01-06 23:20:29 +01:00
Heiner Lohaus
12c413fd2e Add Edge as Browser for nodriver
Fix for RetryProviders doesn't retry
Add retry and continue for DuckDuckGo provider
Add cache for Cloudflare provider
Add cache for prompts on gui home
Add scroll to bottom checkbox in gui
Improve prompts on home gui
Fix response content type in api for files
2025-01-05 17:02:15 +01:00
Heiner Lohaus
7893a0835e Add filessupport, scrape and refine your data
Remove Webdriver usages
Add continue messages for other providers
2025-01-01 04:20:02 +01:00
Heiner Lohaus
6390fccafd Fix unittests 2024-12-18 13:04:50 +01:00
Heiner Lohaus
5b76f2cde0 Fix docker build and fix api_base issue in OpenaiAPI providers 2024-12-16 19:07:41 +01:00
Heiner Lohaus
6a624acf55 Use custom user data dir for each provider
Reuse cookies and access token in Copilot
Send in the gui messages to multiple providers at once
Add GUI documenation
2024-12-07 19:38:04 +01:00
Heiner Lohaus
cab71ca8b6 Add duckduckgo-search to slim requirements,
Add pass provider as model in the client,
Fix missing @property in version utils
2024-11-26 16:36:45 +01:00
Heiner Lohaus
275404373f Add --cookie-browsers argument to cli 2024-11-19 20:45:25 +01:00
Heiner Lohaus
f1ef23285a Fix load model list i AirforceChat provider
Add Microsoft Copilot provider
Show image support in the model list of the gui
2024-11-18 15:41:45 +01:00
H Lohaus
6ce493d4df
Fix api streaming, fix AsyncClient (#2357)
* Fix api streaming, fix AsyncClient, Improve Client class, Some providers fixes, Update models list, Fix some tests, Update model list in Airforce provid
er, Add OpenAi image generation url to api, Fix reload and debug in api arguments, Fix websearch in gui

* Fix Cloadflare and Pi and AmigoChat provider

* Fix conversation support in DDG provider, Add cloudflare bypass with nodriver

* Fix unittests without curl_cffi
2024-11-16 13:19:51 +01:00
Heiner Lohaus
8fcf618bbe Add Ollama provider, Add vision support to Openai 2024-05-05 23:38:31 +02:00
Heiner Lohaus
b7eee50930 Move raise_for_status, Create FormData wrapper 2024-03-15 14:55:53 +01:00
Heiner Lohaus
8cc6000ffb Add Pyinstaller support, Use curl_cffi in You provider 2024-03-15 11:46:06 +01:00