Delete TGBRIDGE.md
This commit is contained in:
1 file changed
-208
-208
@@ -1,208 +0,0 @@
|
|||||||
# Telegram bridge — rebuild notes
|
|
||||||
|
|
||||||
Everything below is in `tgbridge/`. Routes, behaviour and env-var names stay
|
|
||||||
backwards compatible: **your existing `credentials.env` works unchanged.**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. What you need to do
|
|
||||||
|
|
||||||
One optional step, for the avatars you asked for:
|
|
||||||
|
|
||||||
1. Make a channel in your server (e.g. `#tg-avatar-cache`) and deny
|
|
||||||
`@everyone` → *View Channel*.
|
|
||||||
2. Right-click it → **Copy Channel ID**.
|
|
||||||
3. Paste it into the `AVATAR_CACHE_CHANNEL_ID=` line already added at the
|
|
||||||
bottom of `credentials.env`.
|
|
||||||
|
|
||||||
Leave it empty and everything else still works — forwarded messages just keep
|
|
||||||
the default webhook avatar.
|
|
||||||
|
|
||||||
Optional: `pip install lottie` to make animated `.tgs` stickers actually
|
|
||||||
animate. Without it they fall back to a static thumbnail, which is still
|
|
||||||
better than the broken attachment they produce today.
|
|
||||||
|
|
||||||
`main.py` now loads the bridge as one extension:
|
|
||||||
|
|
||||||
```python
|
|
||||||
await bot.load_extension('tgbridge')
|
|
||||||
```
|
|
||||||
|
|
||||||
**Behavioural change worth knowing:** the Telegram poller now starts when the
|
|
||||||
extension loads, not from `on_ready`. That's what fixes the duplicate-poller
|
|
||||||
bug (A3), but it means startup happens before the Discord gateway connects. A
|
|
||||||
Telegram failure at that moment is caught and retried in the background (5
|
|
||||||
attempts, backing off 30 s → 10 min) — it will **not** stop the rest of the bot
|
|
||||||
from booting. Verified with a deliberately invalid token: the bot loaded fine,
|
|
||||||
both cogs registered, and `!bridge status` showed the error.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Bugs fixed
|
|
||||||
|
|
||||||
These were all live defects, not hypotheticals.
|
|
||||||
|
|
||||||
| # | Bug | Effect before |
|
|
||||||
|---|---|---|
|
|
||||||
| A1 | `parse_mode='Markdown'` with an unescaped body | **Every** Discord message containing a stray `*`, `_`, `` ` `` or `[` was rejected by Telegram with a 400 and silently dropped. Now HTML parse mode with proper escaping, plus a plain-text retry if Telegram still complains. |
|
|
||||||
| A2 | No `allowed_mentions` on webhook posts | Anyone in the Telegram group could type `@everyone` and ping your whole server. Now `{"parse": []}` on every post. |
|
|
||||||
| A3 | `forwarder.start()` called from `on_ready` | `on_ready` fires again after every gateway reconnect, starting a second poller on the same token → Telegram 409 Conflict and duplicated messages. Now starts once at extension load, with a `_running` guard. |
|
|
||||||
| A4 | `document.mime_type.startswith(...)` | `mime_type` is nullable; any document without one raised into the bare `except` and the message was dropped. |
|
|
||||||
| A5 | `temp_in_path` used in `finally`, assigned in `try` | If `NamedTemporaryFile` raised, cleanup threw `UnboundLocalError` and masked the real error. |
|
|
||||||
| A6 | ffmpeg cleanup after `wait_for`, no kill on timeout | A conversion timeout leaked the temp file **and** left an orphaned ffmpeg process forever. Now killed and reaped in every path. |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Stickers — measured, not guessed
|
|
||||||
|
|
||||||
Your old code ran all three Telegram sticker formats through one GIF
|
|
||||||
conversion. I tested each:
|
|
||||||
|
|
||||||
| Format | Before | Now |
|
|
||||||
|---|---|---|
|
|
||||||
| **Static `.webp`** | Re-encoded to GIF. Measured: the antialiased edge went from **228 alpha levels to 2**, i.e. a visible jagged halo on every sticker — and the file got **4× bigger** (942 B → 3771 B) and was downscaled 512→320. | Passed through untouched. Discord renders `.webp` natively with full alpha. Zero CPU. |
|
|
||||||
| **Animated `.tgs`** | ffmpeg exits **183 "Invalid data found"** (it's gzipped Lottie JSON, ffmpeg cannot read it). The `returncode == 0` check failed, so the raw `.tgs` was uploaded as an **unopenable junk attachment**. | Rendered via the optional `lottie` library; falls back to Telegram's own static thumbnail + emoji. |
|
|
||||||
| **Video `.webm`** | Converted to GIF, same alpha crushing. | Converted to **animated WebP** — loops in Discord, keeps 8-bit alpha, and measured **7.4× smaller** than the GIF (16 KB vs 122 KB). |
|
|
||||||
|
|
||||||
**One subtle bug worth calling out.** ffmpeg's *default* VP9 decoder silently
|
|
||||||
discards the alpha channel of VP9-in-WebM — every transparent pixel decodes to
|
|
||||||
opaque black. Only `libvpx-vp9`, selected **before** `-i`, reads it. Telegram
|
|
||||||
video stickers are exactly this format. Verified: without the flag, 1 alpha
|
|
||||||
level; with it, 217. This is now handled in `media.py:_decoder_args`.
|
|
||||||
|
|
||||||
Converted stickers are cached in sqlite by `file_unique_id`, so each one is
|
|
||||||
only ever converted once.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Profile-photo avatars
|
|
||||||
|
|
||||||
Telegram's only file URL embeds your bot token
|
|
||||||
(`api.telegram.org/file/bot<TOKEN>/...`), so it can never go in a webhook
|
|
||||||
payload. Instead the photo is re-uploaded once to your private cache channel
|
|
||||||
and that link is used as `avatar_url`.
|
|
||||||
|
|
||||||
I verified against the live API that **Discord re-hosts the image at post
|
|
||||||
time**: `author.avatar` comes back as a fresh hash distinct from the webhook's
|
|
||||||
own. So the source link only needs to be alive at the moment of the POST, and
|
|
||||||
already-forwarded messages keep their avatar forever.
|
|
||||||
|
|
||||||
Caching behaviour, all verified by test:
|
|
||||||
|
|
||||||
- repeat lookups cost **zero** API calls
|
|
||||||
- users with no photo (or privacy-hidden) are cached negatively, so it doesn't
|
|
||||||
re-ask on every message
|
|
||||||
- when the hosted link goes stale it re-uploads **from the stored blob**,
|
|
||||||
without re-downloading from Telegram
|
|
||||||
- when someone changes their photo it's detected and refreshed
|
|
||||||
- channel posts use the chat photo instead of a user photo
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. Features added
|
|
||||||
|
|
||||||
**Reply threading (both directions).** A Telegram reply renders as a Discord
|
|
||||||
quote block with a clickable jump link to the original forwarded message; a
|
|
||||||
Discord reply becomes a native Telegram reply. Backed by a message-ID map in
|
|
||||||
`data/tgbridge.db`.
|
|
||||||
|
|
||||||
**Edit sync (both directions).** Telegram `edited_message` updates patch the
|
|
||||||
Discord message; Discord edits patch the Telegram one. The Discord side uses
|
|
||||||
`on_raw_message_edit`, **not** `on_message_edit` — the cached variant never
|
|
||||||
fires for messages sent before the last restart, so edit sync would have
|
|
||||||
silently worked in testing and failed in production.
|
|
||||||
|
|
||||||
**Delete sync (Discord → Telegram only).** Including bulk deletes. The reverse
|
|
||||||
isn't possible: Telegram's Bot API gives bots no delete events at all.
|
|
||||||
|
|
||||||
**Albums.** Telegram sends an album as N separate updates, so a 5-photo post
|
|
||||||
used to become 5 separate Discord messages. Now debounced (1.5 s) into one
|
|
||||||
message with up to 10 attachments. All N Telegram IDs map to the single
|
|
||||||
Discord message, so a reply to *any* photo in the album still resolves.
|
|
||||||
|
|
||||||
**Message types that used to say `*(unsupported message type)*`:** voice notes
|
|
||||||
(with duration), video notes, audio (with artist/title), polls (question +
|
|
||||||
options), locations and venues (with a maps link), contacts, dice rolls,
|
|
||||||
join/leave notices, pins, and "forwarded from X" attribution.
|
|
||||||
|
|
||||||
**Rich text.** Telegram entities → Discord markdown and back: bold, italic,
|
|
||||||
underline, strikethrough, spoilers, inline code, code blocks with language,
|
|
||||||
links, and blockquotes.
|
|
||||||
|
|
||||||
> Telegram entity offsets are counted in **UTF-16 code units**, not Python
|
|
||||||
> characters, so any message containing an emoji shifts every following offset
|
|
||||||
> and naive slicing corrupts the formatting. The converter works on a UTF-16
|
|
||||||
> buffer and decodes once at the end. Verified with emoji before, inside and
|
|
||||||
> after formatted spans.
|
|
||||||
|
|
||||||
**Discord references resolved.** `<@123>`, `<@&123>`, `<#123>` and
|
|
||||||
`<:emoji:123>` used to forward to Telegram as raw snowflakes; they now become
|
|
||||||
readable names.
|
|
||||||
|
|
||||||
**Long messages split** instead of being truncated at 2000 chars (`content[:2000]`),
|
|
||||||
on the nicest available boundary. Same on the Telegram side at 4096.
|
|
||||||
|
|
||||||
**Rate limits and retries.** One shared `aiohttp` session instead of a new one
|
|
||||||
per message; 429s now wait the `retry_after` Discord asks for; 5xx retries with
|
|
||||||
exponential backoff.
|
|
||||||
|
|
||||||
**Oversized uploads** are detected and posted as a notice naming the file,
|
|
||||||
instead of vanishing.
|
|
||||||
|
|
||||||
**Markdown injection blocked.** Telegram text is escaped before it reaches
|
|
||||||
Discord, and Discord text is HTML-escaped before it reaches Telegram, so
|
|
||||||
neither side can inject formatting or HTML into the other.
|
|
||||||
|
|
||||||
**Route auto-discovery.** Any `TELEGRAM_GROUP_ID_<SUFFIX>` +
|
|
||||||
`DISCORD_WEBHOOK_URL_<SUFFIX>` + `DISCORD_CHANNEL_ID_<SUFFIX>` trio becomes a
|
|
||||||
route automatically — a third bridged channel is now an `.env` edit, not a code
|
|
||||||
change. `data/bridge_routes.json` can add or override routes. Your existing
|
|
||||||
`main` and `shitpost` routes were confirmed to load unchanged.
|
|
||||||
|
|
||||||
**`!bridge status`** — routes, uptime, forwarded counts, webhook success/failure,
|
|
||||||
cached avatars and stickers, whether Lottie is installed, and the last error.
|
|
||||||
|
|
||||||
**Explicit echo guard.** The bridge now checks the sender against its own bot
|
|
||||||
ID. Previously loops were avoided only incidentally, which would have bitten as
|
|
||||||
soon as edit sync or a second token was added.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. Files
|
|
||||||
|
|
||||||
| File | Purpose |
|
|
||||||
|---|---|
|
|
||||||
| `__init__.py` | extension entry point; wires and starts everything |
|
|
||||||
| `config.py` | route discovery from env + optional JSON |
|
|
||||||
| `store.py` | sqlite: message map, avatar cache, sticker cache |
|
|
||||||
| `entities.py` | Telegram entities ↔ Discord markdown ↔ Telegram HTML |
|
|
||||||
| `media.py` | ffmpeg / Lottie conversion, with process kill + cleanup |
|
|
||||||
| `avatars.py` | profile photos → Discord CDN |
|
|
||||||
| `webhook.py` | Discord webhook client: session, retries, splitting |
|
|
||||||
| `forwarder.py` | Telegram → Discord |
|
|
||||||
| `discord_to_telegram.py` | Discord → Telegram |
|
|
||||||
| `status_cog.py` | `!bridge status` |
|
|
||||||
|
|
||||||
New database at `data/tgbridge.db` (created automatically, `data/` is already
|
|
||||||
gitignored). Message mappings are pruned after 30 days.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. What I could not test
|
|
||||||
|
|
||||||
I have no access to your Telegram group or a live Discord channel, so the
|
|
||||||
end-to-end path is **unverified against the real APIs**. What I did verify:
|
|
||||||
|
|
||||||
- every module imports, and `main.py` parses
|
|
||||||
- both directions exercised with mocked Telegram/Discord objects, covering
|
|
||||||
~25 message shapes (text, entities, photos, albums, replies, edits, echo
|
|
||||||
guard, null MIME types, polls, locations, dice, voice, splitting)
|
|
||||||
- ffmpeg conversions run on real generated media, with alpha fidelity measured
|
|
||||||
- avatar caching logic, including staleness and photo-change paths
|
|
||||||
- route discovery against your actual `credentials.env`
|
|
||||||
- the Discord avatar re-hosting question, against the live API (one test
|
|
||||||
message, deleted immediately afterwards)
|
|
||||||
|
|
||||||
Worth watching on first run: whether Telegram accepts the generated HTML for
|
|
||||||
unusual messages (there's a plain-text fallback if it doesn't), and whether the
|
|
||||||
bot has permission to delete messages in the Telegram group for delete sync.
|
|
||||||
Reference in new issue
Block a user