Add self-hosted RouterOS package feed and pinned updater redirect
Check source / check (push) Successful in 12s
Build RouterOS package release / release (push) Successful in 5m14s
Publish and promote RouterOS project site / site (push) Successful in 8s

This commit is contained in:
feedc0de committed 2026-11-22 11:50:00 +01:00
1 parent 37308f1636
commit 9217046b33
9 files changed
+207 -21

No files matched your search

+5
View File
@@ -121,8 +121,13 @@ jobs:
for arch in arm arm64; do
wifi=wifi-qcom-ac
if [ "$arch" = arm64 ]; then wifi=wifi-qcom; fi
update_host_args=""
if [ "$ROUTEROS_VERSION" = 7.25.1 ]; then
update_host_args="--update-host mikrotik.brunner.ninja"
fi
.venv/bin/python tools/build-release.py --version "$ROUTEROS_VERSION" \
--arch "$arch" --wifi "$wifi" --keys "$keyfile" --output "build/release-$arch" \
$update_host_args \
--option-root "build/runtime-$arch/option-root" \
--olsr-root "build/runtime-$arch/olsr-root"
done
+4 -3
View File
@@ -77,9 +77,10 @@ both Linux and Windows Netinstall binaries. You do not need Windows to patch the
Windows executable.
The public site and RouterOS metadata feed are built from `service/` and `site/`;
the Kubernetes chart is in `chart/mikrotik-olsr-site/`. The current feed serves
channel versions and changelogs, while package-download protocol support and the
RouterOS updater-host patch remain under investigation. See
the Kubernetes chart is in `chart/mikrotik-olsr-site/`. The feed serves channel
versions, changelogs, package lists, and signed NPK downloads. The 7.25.1
ARM/ARM64 build can redirect its updater host to the project service; a live
update transaction remains to be verified. See
[update-server.md](docs/update-server.md).
Install prerequisites on Debian/Ubuntu:
+5
View File
@@ -89,6 +89,11 @@ plain-text RouterOS channel and changelog responses. Package download support
and the updater host patch are not yet verified; see
[update-server.md](update-server.md).
For 7.25.1 ARM and ARM64, the release build also redirects the updater host to
`mikrotik.brunner.ninja`. This is still awaiting a live updater transaction;
see [update-server.md](update-server.md). Other versions remain unchanged until
their updater binaries are verified separately.
The service chart workflow publishes an immutable image and chart from `main`,
then promotes the chart version to Argo CD. It needs `PACKAGE_USERNAME`,
`PACKAGE_TOKEN`, `QUAY_USERNAME`, `QUAY_TOKEN`, and `GITOPS_TOKEN`; it receives no
+29 -8
View File
@@ -37,14 +37,35 @@ when publishing a new package version. `/healthz` is used by the Kubernetes
probes. `/api/releases` returns public Gitea release metadata for the landing
page.
These endpoints are a working metadata feed, not yet a complete package updater.
The project has not yet changed `sys2` to query this service, and the package
download request and response format has not been confirmed. A version appearing
in the update dialog must not be treated as proof that the signed NPK download
will work. The test plan is to first patch the updater host in a test system NPK,
capture its requests on a disposable RouterOS device, implement the matching
package response and downloads, then test install and rollback before marking
updates available by default.
The service also serves `/routeros/<version>/packages.csv` in the observed
`arch,name,size` format and streams the matching published, signed NPK at
`/routeros/<version>/<package>-<version>-<arch>.npk`. It selects the newest
published release revision for each package/architecture. The package list and
download were exercised against a public 7.25.1 Gitea release locally.
For the exact official 7.25.1 ARM and ARM64 `sys2` builds, the NPK build can
replace both references to `upgrade.mikrotik.com` with
`mikrotik.brunner.ninja`:
```sh
python tools/routeros.py npk routeros-7.25.1-arm.npk --arch arm \
--patch-system --update-host mikrotik.brunner.ninja \
--keys .keys/development.json --output build/routeros-7.25.1-arm-updates.npk
```
The patch rejects any other `sys2` digest. Both ARM and ARM64 test NPKs built,
self-verified, and contained only the project host in the updater binary.
The release workflow applies it to 7.25.1 builds. **A live RouterOS update
check/download from this patched binary remains unverified.** Do not treat a
version appearing in the dialog as proof that installation works. The exact
package request used by RouterOS and its behavior on the small-flash hAP ac2
still need a device test before a release is advertised as a working updater.
The update dialog handles version changes for installed packages. It does not
offer a removed optional package as a same-version installation. For that
experiment, download the published OLSR NPK from this site into RouterOS Files
and reboot, or upload the same file with Winbox Files; do not use the version
check button to reinstall it.
## Hosting
+80 -6
View File
@@ -3,6 +3,7 @@
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
import json
from pathlib import Path
import re
import sys
from urllib.error import URLError
from urllib.request import Request, urlopen
@@ -15,8 +16,48 @@ if not SITE.is_dir(): # Running from the source tree: service/server.py.
SITE = HERE.parent / "site"
FEED = json.loads((SITE / "feed" / "channels.json").read_text())
RELEASES_URL = (
"https://code.brunner.ninja/api/v1/repos/feedc0de/mikrotik-olsr/releases?limit=10"
"https://code.brunner.ninja/api/v1/repos/feedc0de/mikrotik-olsr/releases?limit=50"
)
PACKAGE_RE = re.compile(r"(routeros|option|olsr|wifi-qcom-ac|wifi-qcom)-"
r"(7\.\d+(?:\.\d+)?)-(arm|arm64)\.npk\Z")
VERSION_RE = re.compile(r"7\.\d+(?:\.\d+)?\Z")
def release_assets(version, releases):
"""Select the newest published revision for each architecture."""
selected = {}
for release in releases:
tag = release.get("tag_name", "")
match = re.fullmatch(rf"ros-{re.escape(version)}(?:-(arm|arm64))?-r(\d+)", tag)
if not match or release.get("draft"):
continue
tag_arch, revision = match.group(1), int(match.group(2))
for asset in release.get("assets", []):
name = asset.get("name", "")
package = PACKAGE_RE.fullmatch(name)
if not package or package.group(2) != version:
continue
arch = package.group(3)
if tag_arch and tag_arch != arch:
continue
url = asset.get("browser_download_url", "")
parsed = urlsplit(url)
if parsed.scheme != "https" or parsed.hostname != "code.brunner.ninja":
continue
if not parsed.path.startswith(
f"/feedc0de/mikrotik-olsr/releases/download/{tag}/"
) or not parsed.path.endswith("/" + name):
continue
key = (arch, package.group(1))
if key not in selected or revision > selected[key][0]:
selected[key] = (revision, asset)
return {key: value[1] for key, value in selected.items()}
def fetch_releases():
req = Request(RELEASES_URL, headers={"User-Agent": "mikrotik-olsr-site"})
with urlopen(req, timeout=8) as response:
return json.load(response)
class Handler(SimpleHTTPRequestHandler):
@@ -41,14 +82,47 @@ class Handler(SimpleHTTPRequestHandler):
return self.send_bytes(200, b"ok\n", "text/plain; charset=utf-8")
if path == "/api/releases":
try:
req = Request(RELEASES_URL, headers={"User-Agent": "mikrotik-olsr-site"})
with urlopen(req, timeout=8) as response:
body = response.read()
json.loads(body)
body = json.dumps(fetch_releases()).encode()
return self.send_bytes(200, body, "application/json; charset=utf-8")
except (URLError, TimeoutError, ValueError) as exc:
return self.send_bytes(502, (str(exc) + "\n").encode(), "text/plain; charset=utf-8")
package_path = re.fullmatch(r"/routeros/(7\.\d+(?:\.\d+)?)/([^/]+)", path)
if package_path:
version, filename = package_path.groups()
if filename == "packages.csv":
try:
assets = release_assets(version, fetch_releases())
except (URLError, TimeoutError, ValueError) as exc:
return self.send_bytes(502, (str(exc) + "\n").encode(), "text/plain")
rows = ["arch,name,size"]
rows += [f"{arch},{name},{asset['size']}" for (arch, name), asset
in sorted(assets.items())]
return self.send_bytes(200, ("\n".join(rows) + "\n").encode(), "text/csv")
package = PACKAGE_RE.fullmatch(filename)
if package and package.group(2) == version:
try:
asset = release_assets(version, fetch_releases()).get(
(package.group(3), package.group(1)))
if asset is None:
return self.send_bytes(404, b"Package not published\n", "text/plain")
req = Request(asset["browser_download_url"],
headers={"User-Agent": "mikrotik-olsr-site"})
with urlopen(req, timeout=30) as response:
length = int(response.headers.get("Content-Length", "0"))
if length != asset["size"]:
raise ValueError("Release asset size mismatch")
self.send_response(200)
self.send_header("Content-Type", "application/octet-stream")
self.send_header("Content-Length", str(length))
self.end_headers()
if self.command != "HEAD":
while block := response.read(65536):
self.wfile.write(block)
return
except (URLError, TimeoutError, ValueError) as exc:
return self.send_bytes(502, (str(exc) + "\n").encode(), "text/plain")
prefix = "/routeros/NEWESTa7."
if path.startswith(prefix):
channel = path[len(prefix):]
@@ -61,7 +135,7 @@ class Handler(SimpleHTTPRequestHandler):
marker = "/CHANGELOG"
if path.startswith("/routeros/") and path.endswith(marker):
version = path[len("/routeros/"):-len(marker)].strip("/")
if version and all(ch.isdigit() or ch == "." for ch in version):
if VERSION_RE.fullmatch(version):
changelog = SITE / "feed" / "changelogs" / f"{version}.txt"
if changelog.is_file():
return self.send_bytes(200, changelog.read_bytes(), "text/plain; charset=utf-8")
+13
View File
@@ -4,6 +4,7 @@ import sys
import time
import unittest
from pathlib import Path
from service.server import release_assets
ROOT = Path(__file__).resolve().parents[1]
@@ -54,3 +55,15 @@ class UpdateServerTests(unittest.TestCase):
status, _, body = self.get("/routeros/NEWESTa7.development")
self.assertEqual(status, 200)
self.assertIn(b"No version available", body)
def test_release_selection_keeps_latest_signed_package_per_architecture(self):
def release(tag, size):
name = "olsr-7.25.1-arm.npk"
return {"tag_name": tag, "draft": False, "assets": [{
"name": name, "size": size,
"browser_download_url":
f"https://code.brunner.ninja/feedc0de/mikrotik-olsr/releases/download/{tag}/{name}"
}]}
result = release_assets("7.25.1", [release("ros-7.25.1-arm-r4", 1),
release("ros-7.25.1-arm-r5", 2)])
self.assertEqual(result[("arm", "olsr")]["size"], 2)
+6 -1
View File
@@ -89,7 +89,11 @@ def build(args):
for source, target in ((linux / 'netinstall-cli', output / 'netinstall-cli'),
(windows, output / 'netinstall.exe')):
run('netinstall', source, '--keys', args.keys, '--output', target)
run('npk', system, '--arch', arch, '--patch-system', '--keys', args.keys, '--output', output / system.name)
system_args = ['npk', system, '--arch', arch, '--patch-system', '--keys', args.keys,
'--output', output / system.name]
if args.update_host:
system_args += ['--update-host', args.update_host]
run(*system_args)
if wifi:
run('npk', wifi, '--arch', arch, '--keys', args.keys, '--output', output / wifi.name)
payloads = {'option': args.option_root, 'olsr': args.olsr_root}
@@ -157,6 +161,7 @@ def main():
help='ARMv7 or ARM64/AArch64 system and add-on packages')
p.add_argument('--wifi', choices=['wifi-qcom-ac', 'wifi-qcom', 'wireless'])
p.add_argument('--keys', required=True)
p.add_argument('--update-host', help='Redirect a verified 7.25.1 system updater to the project feed')
p.add_argument('--output', required=True)
p.add_argument('--option-root', help='Audited runtime payload directory')
p.add_argument('--olsr-root', help='Built OLSR payload directory')
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env python3
"""Redirect the verified 7.25.1 ARM/ARM64 RouterOS updater host.
The two references are ELF32 absolute literal words. The second string has
four trailing NUL bytes, enough for mikrotik.brunner.ninja without moving a
section. The first reference is redirected to that second string.
"""
import argparse
import hashlib
from pathlib import Path
import struct
LAYOUTS = {
"6fa7c71d0ff6014d3d65f3e696893bd87a4b3635b9a06d51de8619524aee9512":
(0x37ce9, 0xfed4, 0x4200c, 0x359c4),
"237ccca17a91d06a52b91d24b4cda60b2a5aca83a47fa1c38719b8b2bba47bea":
(0x384bf, 0x118a8, 0x4286c, 0x35b1c),
}
OLD = b"upgrade.mikrotik.com\0"
SLOT_SIZE = 24
def patch(data: bytes, host: str) -> bytes:
if host != "mikrotik.brunner.ninja":
raise ValueError("Only the verified project hostname is supported")
layout = LAYOUTS.get(hashlib.sha256(data).hexdigest())
if layout is None:
raise ValueError("Unsupported sys2 build; updater host was not changed")
first, first_ref, second, second_ref = layout
if (data[first:first + len(OLD)] != OLD or data[second:second + SLOT_SIZE] !=
OLD + b"\0" * (SLOT_SIZE - len(OLD))):
raise ValueError("Unexpected updater string layout")
if (data[first_ref:first_ref + 4] != struct.pack("<I", first + 0x10000) or
data[second_ref:second_ref + 4] != struct.pack("<I", second + 0x10000)):
raise ValueError("Unexpected updater reference layout")
result = bytearray(data)
result[first:first + len(OLD)] = b"\0" * len(OLD)
result[first_ref:first_ref + 4] = struct.pack("<I", second + 0x10000)
replacement = host.encode("ascii") + b"\0"
result[second:second + SLOT_SIZE] = replacement.ljust(SLOT_SIZE, b"\0")
if OLD in result or result.count(replacement) != 1:
raise ValueError("Updater host replacement did not verify")
return bytes(result)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("source", type=Path)
parser.add_argument("output", type=Path)
parser.add_argument("--host", default="mikrotik.brunner.ninja")
args = parser.parse_args()
args.output.write_bytes(patch(args.source.read_bytes(), args.host))
print(f"Patched updater host in {args.output}")
if __name__ == "__main__":
main()
+7 -3
View File
@@ -17,6 +17,7 @@ from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey,
from npk import NovaPackage, NpkPartID, NpkFileContainer
from toyecc import ECPrivateKey, getcurvebyname
import keypatch
import patch_update_host
OFFICIAL_NPK_KEY = 'c293ced638a2a33c681fc8de98ee26c54eadc5390c2dfce197d35c83c416cf59'
@@ -84,16 +85,18 @@ def pack_squashfs(source, output):
'-b', '256k', '-all-root', '-no-xattrs', '-no-progress', '-quiet'], check=True)
def patch_tree(root, mapping):
def patch_tree(root, mapping, update_host=None):
hits = []
for file in sorted(Path(root).rglob('*')):
if file.is_symlink() or not file.is_file():
continue
data = file.read_bytes()
if update_host and file.relative_to(root).as_posix() == 'nova/bin/sys2':
data = patch_update_host.patch(data, update_host)
updated = data
for old, new in mapping.items():
updated = keypatch.replace_key(old, new, updated, str(file.relative_to(root)))
if updated != data:
if updated != file.read_bytes():
file.write_bytes(updated)
hits.append(str(file.relative_to(root)))
if not hits:
@@ -142,7 +145,7 @@ def npk_build(args):
tree = work / 'root'
image.write_bytes(part.data)
subprocess.run(['unsquashfs', '-no-progress', '-d', str(tree), str(image)], check=True, stdout=subprocess.DEVNULL)
print('Patched system files:', ', '.join(patch_tree(tree, mapping)))
print('Patched system files:', ', '.join(patch_tree(tree, mapping, args.update_host)))
pack_squashfs(tree, work / 'output.sfs')
part.data = (work / 'output.sfs').read_bytes()
changed_system = True
@@ -206,6 +209,7 @@ def main():
if name == 'npk':
p.add_argument('--arch', choices=['arm', 'arm64', 'x86', 'mipsbe', 'mmips', 'smips', 'ppc'], required=True)
p.add_argument('--patch-system', action='store_true')
p.add_argument('--update-host', help='Verified 7.25.1 ARM/ARM64 updater redirect')
p.set_defaults(func=function)
args = parser.parse_args()
args.func(args)