Add gopher documentation to README and execution notes
- Update README with gopher/phlog section, publishing commands, URLs, and first-time SDF setup instructions - Add gopher workflow documentation to execution-notes.txt - Include planning docs in docs/ directory
This commit is contained in:
@@ -0,0 +1,224 @@
|
||||
# Gopher Integration Code Review
|
||||
|
||||
> **Date:** 2026-01-13
|
||||
> **Reviewer:** Claude (Opus 4.5)
|
||||
> **Status:** Ready for deployment
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Implementation of gopher integration for marcus-web is complete. The system converts Hugo blog posts with `phlog: true` frontmatter to gopher-friendly plain text and deploys them to SDF gopherspace.
|
||||
|
||||
**7 commits total** | **All tests passing** | **1 critical bug found and fixed**
|
||||
|
||||
---
|
||||
|
||||
## Files Created
|
||||
|
||||
### scripts/gopher/ascii_art.py
|
||||
ASCII art assets and helper functions for gopher content.
|
||||
|
||||
**Contents:**
|
||||
- Post header (MNW owl logo)
|
||||
- Category headers (Fun Center, Frank's Couch, Beercalls)
|
||||
- Post footer (SDF box)
|
||||
- Series-to-directory mapping
|
||||
- Movie metadata ASCII table generator
|
||||
- Text wrapping utilities
|
||||
|
||||
**Review:** Clean, well-organized. All art fits within 70-char line width constraint.
|
||||
|
||||
---
|
||||
|
||||
### scripts/gopher/convert_to_gopher.py
|
||||
Main conversion script: markdown to gopher text.
|
||||
|
||||
**Key functions:**
|
||||
- `parse_frontmatter()` - YAML extraction
|
||||
- `extract_links()` - Converts `[text](url)` to `text [1]` with footer references
|
||||
- `convert_headings()` - `#` → section headers, `##` → subheadings
|
||||
- `convert_formatting()` - `**bold**` → `*bold*`, `*italic*` → `_italic_`
|
||||
- `convert_code_blocks()` - Fenced blocks → 4-space indented
|
||||
- `handle_imdbposter()` - Hugo shortcode → ASCII movie table
|
||||
- `wrap_paragraphs()` - 70-char line wrapping
|
||||
|
||||
**Review:** Solid implementation. Handles edge cases well. The imdbposter conversion extracts viewing info tables and preserves them as plain text.
|
||||
|
||||
**Fixed bug:** Added `sys.path.insert()` for imports to work when called from shell script.
|
||||
|
||||
---
|
||||
|
||||
### scripts/gopher/generate_gophermaps.py
|
||||
Generates gophermap index files for categories.
|
||||
|
||||
**Outputs:**
|
||||
- `blog/gophermap` - Main index with category links and post counts
|
||||
- `blog/fun-center/gophermap` - Tech posts listing
|
||||
- `blog/franks-couch/gophermap` - Movie reviews listing
|
||||
- `blog/beercalls/gophermap` - Beer logs listing (when posts exist)
|
||||
|
||||
**Review:** Properly formats gophermap syntax with tab-separated fields. Sorts posts by date (newest first). Includes ASCII art headers per category.
|
||||
|
||||
**Fixed bug:** Same `sys.path` fix as convert_to_gopher.py.
|
||||
|
||||
---
|
||||
|
||||
### scripts/gopher/update_root_gophermap.py
|
||||
Generates updated root gophermap with "FROM THE WEB" section.
|
||||
|
||||
**Review:** Contains full gophermap template. One-time use script for initial setup. Includes dynamic `=echo` directives for latest post detection.
|
||||
|
||||
**Note:** This needs to be run manually after first deployment:
|
||||
```bash
|
||||
python3 scripts/gopher/update_root_gophermap.py
|
||||
scp gopher_build/gophermap mnw@sdf.org:~/gopher/gophermap
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
### scripts/remote_publish.sh
|
||||
Added `--gopher` and `--gopher-only` flags.
|
||||
|
||||
**Changes:**
|
||||
- Argument parsing with help text
|
||||
- Conditional Hugo deployment (skipped with `--gopher-only`)
|
||||
- Gopher build: runs convert + generate scripts locally
|
||||
- rsync with `--delete` to sync to SDF
|
||||
- Permission fixing on remote (644 files, 755 dirs)
|
||||
|
||||
**Review:** Good error handling via `set -e`. Clear output messages. The `--delete` flag means removing `phlog: true` from a post will remove it from gopher on next deploy.
|
||||
|
||||
---
|
||||
|
||||
### scripts/new_techpost.py
|
||||
Added phlog prompt after summary input.
|
||||
|
||||
**Changes:**
|
||||
- New prompt: "Publish to gopher phlog? (y/N)"
|
||||
- Added `phlog` parameter to `create_post()`
|
||||
- Frontmatter now includes `phlog: true/false`
|
||||
|
||||
**Review:** Clean integration. Defaults to "no" which is appropriate.
|
||||
|
||||
---
|
||||
|
||||
### scripts/import_letterboxd.py
|
||||
Added phlog prompt after viewing details.
|
||||
|
||||
**Changes:**
|
||||
- Prompt added in `import_movie()` function
|
||||
- `phlog` parameter passed to `create_draft_post()`
|
||||
- Frontmatter updated
|
||||
|
||||
**Review:** Good placement - asks after all other inputs are collected.
|
||||
|
||||
---
|
||||
|
||||
### scripts/new_movie.py
|
||||
Added phlog prompt after TMDB fetch.
|
||||
|
||||
**Changes:**
|
||||
- Same pattern as other scripts
|
||||
- Frontmatter includes phlog field
|
||||
|
||||
**Review:** Consistent with other scripts.
|
||||
|
||||
---
|
||||
|
||||
### .gitignore
|
||||
Added `gopher_build/` to ignore list.
|
||||
|
||||
---
|
||||
|
||||
### requirements.txt
|
||||
Added `pyyaml` dependency.
|
||||
|
||||
---
|
||||
|
||||
## Content Changes
|
||||
|
||||
Three posts enabled for gopher with `phlog: true`:
|
||||
- `content/posts/blog-posting.md` (Fun Center)
|
||||
- `content/posts/recovering-failed-ubuntu-upgrade.md` (Fun Center)
|
||||
- `content/posts/megalopolis.md` (Frank's Couch)
|
||||
|
||||
---
|
||||
|
||||
## Bug Found and Fixed
|
||||
|
||||
### Critical: Import Path Issue
|
||||
|
||||
**Problem:** `from ascii_art import ...` failed when scripts were invoked from `remote_publish.sh` because Python couldn't find the module.
|
||||
|
||||
**Root cause:** When running `python3 scripts/gopher/convert_to_gopher.py` from the project root, Python's working directory doesn't include `scripts/gopher/`.
|
||||
|
||||
**Fix:** Added to both convert and generate scripts:
|
||||
```python
|
||||
import sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
```
|
||||
|
||||
**Commit:** `b5bb06e`
|
||||
|
||||
---
|
||||
|
||||
## Minor Issues Fixed
|
||||
|
||||
1. **No-op regex removed** - `re.sub(r"_([^_]+)_", r"_\1_", text)` did nothing
|
||||
2. **Unused variable removed** - `in_code_block = False` was never used
|
||||
|
||||
---
|
||||
|
||||
## Testing Performed
|
||||
|
||||
1. **Conversion test:** `python3 scripts/gopher/convert_to_gopher.py --all` - 3 posts converted
|
||||
2. **Gophermap test:** `python3 scripts/gopher/generate_gophermaps.py` - All maps generated
|
||||
3. **Output verification:** Manually inspected generated .txt files and gophermaps
|
||||
4. **Import fix verification:** Re-ran after fix, all working
|
||||
|
||||
---
|
||||
|
||||
## Deployment Checklist
|
||||
|
||||
- [ ] `git push` to sync with origin
|
||||
- [ ] Run `./scripts/remote_publish.sh --gopher` for first deployment
|
||||
- [ ] Run `python3 scripts/gopher/update_root_gophermap.py` to generate root map
|
||||
- [ ] `scp gopher_build/gophermap mnw@sdf.org:~/gopher/gophermap` to update root
|
||||
- [ ] Verify at `gopher://sdf.org/1/users/mnw/blog/`
|
||||
- [ ] Test via web proxy: https://gopher.floodgap.com/gopher/gw?a=gopher://sdf.org/1/users/mnw/blog/
|
||||
|
||||
---
|
||||
|
||||
## Future Considerations
|
||||
|
||||
1. **Beercalls:** No beercall posts have `phlog: true` yet. The category exists but is empty.
|
||||
|
||||
2. **Movie metadata:** The `megalopolis.md` post doesn't have `year`, `director`, `runtime`, or `genres` in frontmatter, so the ASCII movie table only shows title and web link. Consider backfilling metadata for better gopher output.
|
||||
|
||||
3. **Link handling:** External links become numbered references. Very long URLs may look awkward in the links section but are functional.
|
||||
|
||||
4. **Line wrapping:** Code blocks preserve formatting. Very long single words won't wrap (expected behavior).
|
||||
|
||||
---
|
||||
|
||||
## Commit History
|
||||
|
||||
```
|
||||
b5bb06e Fix import path issue in gopher scripts
|
||||
2f20e02 Add pyyaml to requirements.txt (Phase 6)
|
||||
96e350c Add phlog: true to existing posts for gopher (Phase 5)
|
||||
2941c7b Add phlog prompt to post creation scripts (Phase 4)
|
||||
99495eb Add gopher deployment to remote_publish.sh (Phase 3)
|
||||
468d238 Add gophermap generation scripts (Phase 2)
|
||||
c4c18ed Add gopher integration core infrastructure (Phase 1)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verdict
|
||||
|
||||
**Ready for deployment.** All functionality implemented per the integration plan. One critical bug was found during review and fixed. Code is clean, well-structured, and tested.
|
||||
Reference in New Issue
Block a user