titlebot-ng-ng/README.md

# titlebot-ng
A Matrix chat bot for Jupiter Broadcasting's "The Lunch" and "Linux Unplugged" chat rooms that collects and posts title suggestions with web-based voting.

### From the Owner
This has gotten out of hand. I've been working 7 days a week for a couple years now and this has been my most recent mortar between the bricks of time that I've been squeezing out of my life between sleep. Simplification needs to happen whenever possible.

# Technical Outline

The project is a mix of Python (Matrix bot services) and Go (voting backend). Each room gets its own instance of every Python service, so the number of running processes scales with the number of configured rooms.

Services:
- **scraper.py** - Reads configured target Matrix room and collects suggestions
- **post_results.py** - Listens for the `!gettitles` command and posts suggestions for voting
- **get_user_list.py** - Collects the list of users in the room (with post counts and last activity) for vote validation
- **get_votes.py** - Listens for the `!getvotes` command and posts vote results to the room
- **jsonEndpoint/** - Go service (port 9080) exposing JSON endpoints for submissions and users
- **voteServer/** - Go service (port 9081) serving the web voting interface

Everything is launched by `run_all.sh`, which auto-installs Go, creates the Python venv, discovers room configs, builds the Go services, and starts one instance of each Python service per room.

## Features
- **Suggestion Collection**: Monitors chat for messages starting with `!suggest`, `!Suggest`, `!suggestion`, `!Suggestion`, `SUGGEST`, or `!sug`
- **Interactive Voting**: Posts suggestions individually with 👍 reactions for voting
- **Web Voting Interface**: Dedicated voting server with validation (exact display-name match + activity window)
- **Multi-Room Support**: One config file per room under `configs/`, with URL slugs for routing
- **Command System**:
  - `!gettitles` - Triggers posting of collected suggestions
  - `!mayhem` - Generates chaotic text using Ollama AI
  - `!countvotes` - Counts 👍 upvotes on posted suggestions
  - `!getvotes` - Posts current results from the voting server
  - `!voteurl` - Shares the voting URL where users can cast their votes
- **Persistent Storage**: Per-room SQLite databases with duplicate prevention
- **Acknowledgment System**: Reacts with ✅ to confirm suggestion collection
- **Command Deduplication**: All commands are logged to prevent duplicate executions

## Quick Start
1. Clone the repo and copy the example config for each room you want to monitor:
   ```bash
   cp configs/config_example.yaml configs/config.yaml
   # Edit configs/config.yaml with your Matrix credentials and room ID
   ```

2. Run everything with a single command (installs Go and creates the venv if missing):
   ```bash
   ./run_all.sh
   ```

   Or run the components individually:
   ```bash
   python3 -m venv .venv && source .venv/bin/activate
   pip install matrix-nio pyyaml requests
   cd jsonEndpoint && go build -o jep . && ./jep
   cd voteServer && go build -o vs . && ./vs
   ```

## Configuration
One YAML file per room in `configs/`. Example (`configs/config_example.yaml`):

```yaml
username: "magbot"
password: "your_bot_password"
room_id: "!your_room_id:homeserver.com"  # Matrix room to monitor
lookback_seconds: 86400  # 24 hours
lookback_limit: 400      # Max messages to look back when scraping
dump_mode: False         # Dump raw messages when scraping (debugging)
dbName: "my_db_name"     # Optional, auto-derived from room_id if empty
vote_host: "localhost"   # Hostname for voting interface (e.g., "example.com", "vote.mysite.xyz")
# vote_path: "my-room-slug"  # Optional URL slug; auto-derived from room name if unset
vote_activity_hours: 72  # Hours a user must have been active within to be eligible to vote
```

The bot resolves each room's voting page to a URL slug (`vote_path` config override, otherwise derived from the room name/alias) and persists the mapping to `room_slugs.json`, which the Go services read for routing.

## Architecture
- **`room.py`**: Core `Room` class handling Matrix connection, database operations, and command processing
- **`post.py`**: `Post` class representing individual suggestions
- **`scraper.py`**: Main suggestion collection loop
- **`post_results.py`**: Suggestion posting service that waits for `!gettitles` commands
- **`get_votes.py`**: Vote results posting service that waits for `!getvotes` commands
- **`get_user_list.py`**: User list collection service for vote validation
- **`jsonEndpoint/`**: Go service providing JSON API for submissions and users (port 9080)
- **`voteServer/`**: Go web server for voting interface (port 9081)
- **`customOllama/`**: Custom Ollama model for mayhem text generation
- **`run_all.sh`**: Master script to start all services

## Database Schema
Per room (suffixed with a sanitized room ID):
- **suggestions.db**: Collected suggestions with metadata (`is_posted`, `event_id`)
- **users.db**: User info (`display_name`, `num_posts`, `last_active`) for vote validation, plus a `fetch_log` of collection runs
- **votes.db**: Web votes cast via the voting interface

Shared:
- **last_run.db / last_post.db**: Timestamps for state management
- Command log tables (`gettitles`, `mayhem`, `countvotes`, `getvotes`, `voteurl`) live inside each room's suggestions database and prevent duplicate executions

## Dependencies
- **Python**: matrix-nio, pyyaml, requests, sqlite3 (built-in)
- **Go**: github.com/mattn/go-sqlite3, gopkg.in/yaml.v2

## Optional: Ollama Integration
For `!mayhem` command functionality:
1. Install [Ollama](https://ollama.ai/)
2. Build the custom model: `ollama create mad_cat_man:latest -f customOllama/Modelfile`
3. Ensure Ollama is running on `localhost:11434`

## Notes
- Each room gets its own SQLite database for suggestions
- The bot acknowledges collected suggestions with ✅ reactions
- Voting uses 👍 reactions on individual suggestion posts and a web voting interface
- Command deduplication prevents spam and duplicate executions
- Safe database naming handles special characters in room IDs
- Web votes are tied to a user's exact Matrix display name; voters must have been active in the room within `vote_activity_hours` of the most recent room activity

# TODO:
- Rewrite in Rust
- Rewrite in Zig
- Rewrite in GO
<s>- Lookback is being a PITA. Needs work </s>