Files
mikrotik-olsr/docs/native-subsystem-research.md
T
feedc0de 37308f1636
Check source / check (push) Successful in 12s
Publish and promote RouterOS project site / site (push) Successful in 15s
Document native RouterOS package and console probes
2026-11-22 07:52:15 +01:00

23 KiB

Reverse-engineering a native OLSR subsystem

The target is /olsr configuration shared by RouterOS console, API, Winbox, and WebFig, with enabled instances restored after reboot and learned routes owned by OLSR. This page records observations from RouterOS 7.25.1 ARM and the work needed to turn them into a working package. It is not an installation guide for a native subsystem yet.

Reproduce the current findings

Extract the official 7.25.1 ARM system and advanced-tools NPKs with the build tooling. Analyze the exact binaries listed here; offsets are not portable to another release or architecture. The 7.25.1 ARM libumsg.so SHA-256 is c365418b1371e1bbc3d07bd87db3b56b4011c16e5923d7f89e2de10a4560fb40.

python3 tools/inspect-routeros-handlers.py \
  build/platform-roots-7.25.1/arm/lib/libumsg.so --match netwatch
python3 tools/inspect-routeros-handlers.py \
  build/platform-roots-7.25.1/arm/lib/libumsg.so --match ospf
readelf -Ws build/platform-roots-7.25.1/arm/bndl/advanced-tools/nova/bin/netwatch \
  | c++filt | rg 'Looper::addHandler|nv::Store'

The inspector parses two fixed tables in this library. The process table has 138 entries of 28 bytes at file offset 0x824b8; the handler table has 296 entries of 56 bytes at 0x833d0. nv::handler_info::prog and nv::handler_info::handler iterate those tables with the same strides. Handler records contain a pointer to their process record, internal and console-path strings, and an address encoded with a one-based byte for each component. The 138 process records are all occupied in this image. Their IDs are not a simple contiguous array: 136 distinct IDs appear, with ID 148 shared by three test-related process names. The Netwatch process record is the first record (0x824b8 in the ARM library, ID 51), while its handler record is number 153 (0x85548, handler ID 1). The handler_info::each process walker ends at a compiled 0xf18 byte span (138 * 28), rather than scanning for a sentinel. A new process cannot be installed by appending a record to the existing file or choosing an unused numeric ID alone: the iteration bound, table placement, name storage, and loader behavior must all agree. Reusing an optional package's ID would be a collision, not a valid extensibility mechanism for a public OLSR release.

The result includes:

Path Process Internal handler Console path
[51,1] netwatch netwatch2 /tool/netwatch
[44,21] route r5/routing/route /ip/route
[44,109] route r5/routing/ospf/instance No separate path in this record

These are static entries inside libumsg.so. A new daemon binary and an NPK filename alone cannot make [new-process,new-handler] routable; the loader and all clients of this table must agree on the new address. The inspector is read-only and rejects a library with any other SHA-256.

An on-device prototype demonstrates the ABI of the exported registry lookups. The signed, disposable olsr-probe package supplies registry-probe and a small preload library from its /pckg/olsr-probe/bin path. The preload library copies Netwatch's process and handler records, substitutes a new process number 173 and handler number 1, and exposes them only to the probe process. On the hAP ac2, the unmodified probe reported 138 processes and 296 handlers with no OLSR lookup; with LD_PRELOAD it reported 139 and 297 and resolved olsr by process number/name and /olsr by handler address/path. Source is in research/olsr-registry-preload.c and research/registry-probe.c. This proves the specific function-reference and lookup ABI on the test device.

The follow-up package put a proxy libumsg.so in its lib directory. The proxy links to a byte-identical copy of the official library named libumsg-base.so and exports the registry lookup overrides above. The actual RouterOS loader and parser loaded these package libraries at boot. A probe run without LD_PRELOAD then observed 139 process records, 297 handler records, and successful OLSR lookups by number and name. The AP remained reachable after reboot. This demonstrates that an installed package can make the registry extension visible system-wide on this exact 7.25.1 ARM image. The cloned record still carries Netwatch's optional-package feature gate, so it is intentionally not used to launch an OLSR daemon. There is no native OLSR handler or persistent object behind the record yet.

The same disposable package also showed why a RAM-only executable smoke test was misleading: a binary that runs from /pckg/olsr-probe/bin failed after copying it to /ram or RouterOS Files, while its dynamically linked probe and preload constructor both worked from the package path. The live loader's environment includes /pckg/olsr-probe/lib in LD_LIBRARY_PATH when that directory exists in the installed NPK. The package-provided proxy test above verified that the loader uses it; this is an observed behavior on the test AP, not a supported RouterOS extension API.

The 7.25.1 ARM loader imports nv::handler_info::each and nv::handler_info::prog from libumsg.so. Decompiling its startup function at 0x1cc4c shows two handler_info::each passes and a feature_info::each pass before it enters the run loop. This confirms that loader consumes the same compiled metadata at startup. We have not yet proven every field of a process record or whether any supported overlay can extend this table. Its child-launch function at 0x1e010 builds a path from the process name under /nova/bin/, passes it to nv::findFile, and invokes execl on the resolved path. A package can therefore supply a process executable in its filesystem, but the loader still needs a registered process record before it can address and launch that executable. This was observed in the ARM binary; the path resolution rules of nv::findFile still need a separate check.

The 7.25.1 ARM64 package also uses an ELF32 ARM libumsg.so. Its SHA-256 is ae52fce75f030d4b4fc6fe9d509532282919fc6d16ed31175378a2771a998505; the same table layout appears at offsets 0x84444 and 0x85360. The inspector accepts that exact binary too and reports the same Netwatch [51,1] and /ip/route [44,21] addresses. This checks one ARM64 release image; it does not establish compatibility with other versions.

Console commands and persistent configuration

Official add-on packages contain nova/lib/console/<decimal-base>.mem files. For example, advanced-tools ships 1166016512.mem (0x45800000) and the system package ships 1073741824.mem (0x40000000). Both start with the same four-byte marker 9e 69 e8 6a, then contain base-relative pointers and command strings. The Netwatch image contains netwatch, print, set, remove, and field names. The parser binary references /nova/lib/console and .mem. At parser address 0xa03e4 it finds console directories; near 0xa051c it parses the decimal base from the .mem filename. Its loader at 0x45bd0 opens the image, obtains its size, and calls mmap(base, size, PROT_READ, MAP_SHARED|MAP_FIXED, fd, 0). This explains the absolute pointers in the file: Netwatch's 0x45800e94 points into an image mapped at 0x45800000. The independent, source-available routeros-extract console decoder successfully decoded this exact 7.25.1 ARM image using the matching parser ELF. It recovered 3 menus, 21 commands, and 179 parameters, including /tool/netwatch/{add,print,set,remove,enable,disable} and fields such as host and disabled. Reproduce this in the ignored build tree:

git clone https://github.com/naterator/routeros-extract.git build/routeros-extract
git -C build/routeros-extract checkout de2a0e51c79fe8263f1d04881d61270aa74c1f61
(cd build/routeros-extract && go build -o ../routeros-extract-bin ./cmd/routeros-extract)
build/routeros-extract-bin console \
  build/platform-roots-7.25.1/arm/bndl/advanced-tools/nova/lib/console/1166016512.mem \
  --parser build/platform-roots-7.25.1/arm/nova/bin/parser \
  -o build/console-netwatch

The tool decodes existing images; it does not generate a complete new image. Command dispatch IDs, validators, and some value tables remain unresolved in its documentation. As a parser-only experiment, research/make-olsr-console-probe.py copies this exact checked image to an unused base (0x45c00000, filename 1170210816.mem), rebases its aligned pointers, moves the Netwatch menu under the root, and renames it olsr. The decoder reports a root /olsr menu for the result. After installing this image in the disposable olsr-probe NPK, the hAP ac2 booted and accepted /olsr/print and /olsr/print detail without a console syntax error. Both showed no objects. The copied commands still have Netwatch parameters and dispatch metadata; no OLSR object store or service is attached. Do not use this image as an OLSR implementation or perform mutating commands through it.

The decoded Netwatch add command is at mapped address 0x458020b8 and has 37 second-position parameters, including name, host, disabled, interval, and timeout. Its set command is at 0x458020ec: it adds a first-position numbers selector and reuses the 37 editable parameters. print at 0x4580203c has 26 display/query parameters, including where, proplist, and detail. These addresses and the parameter lists can be inspected in build/console-netwatch/1166016512/nodes.json. They are parser metadata, not the daemon's configuration store. A new OLSR menu would need matching command metadata and a handler that implements the operations.

The Netwatch binary registers an nv::Handler through nv::Looper::addHandler(1, ...), matching its table address [51,1]. It uses nv::Store and references /nova/store/netwatch2 and /nova/store/netwatch. Headless Ghidra analysis of its startup function at 0x1c278 confirms it creates an AMap, registers the handler, reads saved messages from nv::Store, and runs the event loop. The reproducible decompiler script is research/ghidra/DumpFunction.java. Its dynamic symbol table imports AMap::cmdAddObj, cmdSetObj, cmdRemoveObj, cmdGetAll, and changed, as well as nv::Store::get and remove, nv::Handler::loadPermData/savePermData, and nv::Looper::addHandler. This identifies the object-management and persistence interfaces used by this particular official handler. It does not yet determine the message field IDs, validation callbacks, or the lifetime of each saved object. The sstore binary contains /rw/store/cfg.%04x and /rw/store/backup strings; those are storage paths, not a documented schema for writing directly to the files.

The generic AMap implementation is in the matching 7.25.1 ARM libubox.so (SHA-256 a4ccd90ceb72fbf9f3f1d92dc631b04bd0725c8d8df60a40b04f5df9571f3f5f). The extracted 7.25.1 ARM64 package contains a byte-identical libubox.so, so this specific AMap implementation is shared by both release images. Its cmdAddObj path creates and validates an object, inserts it into a map, calls storeObj for persistent objects, and notifies subscribers. cmdSetObj validates a replacement, then stores or removes the previous persistent representation and notifies subscribers. cmdRemoveObj removes the object and its saved representation. AMap::init iterates saved IDs using nv::Store::get and calls restoreObj; saveObj calls nv::Store::set, and removeObj calls nv::Store::remove. restoreObj runs validation before adding the object to the live map and discards an invalid saved object. These are decompiled control-flow observations; Ghidra's inferred C parameter names and register assignments in these functions are unreliable. The call sequence is much firmer than a reconstructed C++ ABI or object layout.

AMap constructs an nv::Store only when its constructor's store argument is nonzero. It defaults to automatic persistence. With auto-store enabled, storeObj calls saveObj immediately, which serializes the object into an nv::message and invokes nv::Store::set. With auto-store disabled, changes are queued; setAutoStore(true) calls flushConfig, which writes or removes the queued objects. This distinction matters for a native instance editor: runtime changes should follow a committed object, rather than a parsed draft. AMap::cmdAddObj and cmdSetObj call notifyChange after a successful store/update path. Decompiling nv::Handler::notifyChange in libumsg.so at mapped addresses 0x43b38 and 0x45fe4 shows it collecting changed object IDs and optional message data for registered subscribers, then requesting or scheduling a notification flush. This is the client refresh mechanism; it is not itself a service reload callback. The AMap add/set/remove paths also call derived-class virtual methods after a successful update. Those are the candidate runtime hooks to investigate for OLSRd regeneration, and their signatures and ordering still need proof.

The Netwatch startup function at 0x1c278 allocates an AMap, registers a handler with nv::Looper::addHandler, and later runs the looper. It also constructs a separate nv::Store and iterates saved messages. The binary contains both /nova/store/netwatch2 and /nova/store/netwatch strings; the second store loop may be migration/compatibility code, but its purpose and store-name assignment have not yet been proven. An OLSR implementation should not copy this second loop without understanding it.

research/amap-probe.c, run from the disposable NPK on the hAP ac2, constructed and destroyed an AMap with an empty store name successfully. This checks the constructor and destructor calling convention only. It does not establish the virtual-method ABI, attach a handler, or persist data.

/opt/ghidra/support/analyzeHeadless /tmp/ghidra-olsr libubox-research \
  -import build/platform-roots-7.25.1/arm/lib/libubox.so \
  -scriptPath research/ghidra \
  -postScript DumpFunction.java 0x18588 0x18a28 0x18e38 0x197d0 \
    0x18520 0x1848c 0x189c8 0x193ac 0x19618 0x19190 \
    0x192e8 0x1939c -deleteProject

This explains the required lifecycle for an OLSR instance: save validated fields through its own registered handler, restore them at boot, and use a derived handler's successful-change hook to regenerate the running daemon configuration. notifyChange separately keeps subscribed UI/API clients current. None of this justifies writing directly to /rw/store or treating a scheduler row as an OLSR object.

We also decompiled the matching 7.25.1 ARM libumsg.so at ELF symbol addresses 0x5ec74 (nv::RosCfg::eachHandler), 0x5ed28 (nv::handler_info::each for processes), 0x5e3f8 and 0x5e5a0 (nv::InterposeRosCfg object callbacks), and 0x40048, 0x402f8, 0x40624 (nv::Store constructor, get, set). Ghidra maps this shared library at base 0x10000, so add that base to these addresses when using the script. RosCfg::eachHandler merely delegates to the fixed handler table; handler_info::each walks the fixed 138-process table in 28-byte strides. InterposeRosCfg forwards object operations through callback pointers. None of these methods by itself registers a new process or persists an OLSR object. nv::Store::set flattens a message and passes its bytes to StoreCord::sendUpdate, with a 64-KiB upper bound in this build. That is a concrete persistence path to study in an existing handler, not a public schema or a safe API for arbitrary third-party objects.

mkdir -p /tmp/ghidra-olsr
/opt/ghidra/support/analyzeHeadless /tmp/ghidra-olsr libumsg-research \
  -import build/platform-roots-7.25.1/arm/lib/libumsg.so \
  -scriptPath research/ghidra \
  -postScript DumpFunction.java 0x6ec74 0x6ed28 0x6e3f8 0x6e5a0 \
    0x50048 0x502f8 0x50624 0x43b38 0x45fe4 -deleteProject

A native OLSR implementation therefore needs a registered process/handler, console metadata with an OLSR object schema, and a persistent backend using the same message and store machinery. The intended instance fields are name, disabled state, interfaces, node address, accepted prefixes, HNA prefixes, routing table, and protocol settings. On each accepted change and at boot, the handler should validate the object and compile an OLSRd configuration into RAM atomically. The current flash/olsrd.conf and scheduler row do none of this.

Winbox and WebFig

The official advanced-tools UI resource is home/web/webfig/advtool-1b2cf0d2b54e9429.jg.gz, indexed by advtool.info. Its Netwatch map has path:[51,1]. That agrees with the libumsg.so handler table and the Netwatch binary registration. The .jg resource declares forms, fields, tabs, actions, and lookup sources; it is the client side of an existing handler. The OLSR .jg should point at the OLSR handler after that handler is registered and tested. A descriptor pointing at the scheduler handler can only edit scheduler objects and cannot create native OLSR config or logs. The Netwatch map has Host, Status, Up, Down, and Test tabs; editable fields include a host address and a type enumeration, while status counters are ro:1. It uses optional and conditional groups, plus built-in enable and comment actions. A certificate selector looks up another handler at path:[19,1]. These are concrete UI building blocks for an OLSR instance editor, but a dropdown cannot discover OLSR interfaces or routing tables until the corresponding backend fields and lookups exist.

Routes and logs

The route process registers many handlers, including OSPF and /ip/route. Its libumsg.so entries reveal the RouterOS route control surface, but we have not identified the internal message/ABI for adding and withdrawing a route under a new protocol owner. Linux netlink routes from OLSRd do not appear in RouterOS's route database on the tested device. This route owner and its protocol label must be traced before deleting the BGP adapter. On the live test AP, a read-only RouterOS query on 2026-11-21 found 699 routes with comment OLSR. One representative row had flags DAb (dynamic, active, BGP), destination 10.11.12.13/32, routing-table=main, and gateway=10.12.4.172%funkfeuer-olsr. The %interface gateway syntax already pins the real next-hop to the mesh interface; it does not change the route's BGP owner. This is an observation of the current adapter, not evidence of a native OLSR route API. On a device where the experimental adapter is running, inspect the same RouterOS view without changing route state:

/ip/route/print detail count-only where comment=OLSR
/ip/route/print detail where dst-address=10.11.12.13/32

The second destination is only an example from the test AP; select a route that exists on the device under inspection. An OLSR-owned implementation must show its own dynamic protocol source and withdraw rows on disable, including when the next hop disappears, before the BGP adapter can be removed.

The 7.25.1 ARM base WebFig resource home/web/webfig/roteros-57fef61b6e4007d5.jg.gz renders the route table's protocol badges from field u112 with a numflag map. The map includes bit 6 as OSPF/o, bit 8 as BGP/b, and bit 26 as ISIS/i; it has no OLSR entry. That proves the visible badge is a client-side interpretation of a route field. Adding OLSR to the descriptor alone would not cause route to set a bit, create a dynamic route, or withdraw one. The route process's source/type encoding and insertion API remain to be identified, and any new flag must be checked for conflicts across architectures and versions.

Likewise, the log process and its topic definitions need analysis before an olsr log topic and per-instance log view can be claimed. RAM ring files are only the current daemon output mechanism. The package contains /nova/etc/lognames, a C++ log::Topic enum with OSPF, BGP, RIP, NETWATCH, and other topics but no OLSR topic. Its comment says to append new IDs and never delete old ones. That identifies the numeric topic source; it does not establish where the running log process registers topic names or how a new daemon should emit them.

Next implementation checkpoints

  1. Trace how loader obtains the process table and how a new process can be registered in a versioned system image without changing existing IDs.
  2. Decode the .mem schema by comparing Netwatch console records with its handler and the Netwatch .jg; generate and load a minimal harmless OLSR print command on disposable hardware.
  3. Reconstruct the nv::Handler/AMap and nv::Store ABI with an isolated ARM binary; validate create, set, remove, reboot restore, and console/API access before connecting OLSRd.
  4. Trace the route process's dynamic protocol insertion and withdrawal path. Verify OLSR route ownership and table selection on a disposable router.
  5. Bind the native .jg resource and logs to the proven handler, then test Winbox and WebFig with multiple OLSR instances and route withdrawal.

Margin Research's RouterOS IPC research is a useful conceptual starting point. It describes loader, Nova messages, and handlers for an older RouterOS generation. In particular, its message walkthrough shows that SYS_TO addresses a process and optional handler, SYS_CMD selects an operation, and loader forwards the request to that process's nv::Looper. This explains why a UI resource with path:[51,1] reaches Netwatch rather than becoming a new subsystem. The article also identifies /nova/etc/loader/system.x3 as a process list in its target build. Our extracted 7.25.1 ARM system package has no loader system.x3; it does have other .x3 files under /nova/etc, and its loader binary advertises -t [filename] to trace messages (default /loader.trace) and -d for a debug dump. The trace option is a promising way to capture a Netwatch request on disposable hardware, but we have not enabled it or determined how to restart the boot-time loader safely. We must trace how this version obtains its process configuration before assigning a new ID; the 2022 layout does not prove how 7.25.1 boots packages. Its Winbox authentication and message-tracing material can guide an authorized local trace, but it does not document a ready-made OLSR UI or console schema. The offsets, tables, and paths above come from this project's 7.25.1 ARM binaries and must be independently verified for other versions. The ARM64 handler table check above is limited to the same 7.25.1 release.

To reproduce the loader trace statically, import the exact ARM loader into headless Ghidra and run FindReferences.java on the strings at 0x2133a (reading configuration) and 0x213b0 (/loader.trace). The resulting references lie in functions 0x1cc4c and 0x1d304, which can then be passed to DumpFunction.java; 0x1e010 is the child-launch function described above. These addresses are specific to this binary.