diff --git a/.gitignore b/.gitignore index d89fcaab..ffe1687b 100644 --- a/.gitignore +++ b/.gitignore @@ -32,6 +32,7 @@ tools/bbbconfgen/test/test-output.conf /_site /docs/.sass-cache /docs/_site +docs/.jekyll-cache/* # Ignore the binary build outputs (except READMEs) /bin/go/* diff --git a/docs/customapps/middleware/base-streaming-service-info-changes-to-frontend_sequencediagram-org.svg b/docs/customapps/middleware/base-streaming-service-info-changes-to-frontend_sequencediagram-org.svg new file mode 100644 index 00000000..290ffd21 --- /dev/null +++ b/docs/customapps/middleware/base-streaming-service-info-changes-to-frontend_sequencediagram-org.svg @@ -0,0 +1 @@ +Streaming ServiceInfo changes to the FrontendFrontendAppMiddlewarePrometheusPrometheus Scraperquery last scrape timestamptimestamp: 2compare with saved timestampequal:do nothingwait 5 secondswrite new datatimestamp: 3query last scrape timestamptimestamp: 3compare with saved timestampnot equal:1. safe timestamp = 32. check if ServiceInfo changedquery ServiceInfo data(multiple querys)data ServiceInfocheck if ServiceInfo data changed- not changed: skip- changed: fire eventnotify ServiceInfo changedwait 5 secondsnotify ServiceInfo changedGET /service-infowrite new datatimestamp: 4GetServiceInfo RPCReturn cachedGetServiceInfoResponseGetServiceInfoResponseRedraw \ No newline at end of file diff --git a/docs/customapps/middleware/middleware-communication.md b/docs/customapps/middleware/middleware-communication.md index 0030b634..8465fbbc 100644 --- a/docs/customapps/middleware/middleware-communication.md +++ b/docs/customapps/middleware/middleware-communication.md @@ -7,4 +7,47 @@ nav_order: 300 --- ## Middleware: Communication -(TODO)0xB10C + +### BitBoxApp <-> Middleware RPCs + +TODO + +The Middleware handles the communication between the BitBoxApp and the BitBoxBase. + +- minimize traffic +- RPC Client/Server (Go RPC package) +- Why Websockets +- End to End encryption +- Noise paring / handshake +- TLS vs Noise +- JWT Authentication + +Specific RPC sequence diagrams: + +* [Streaming ServiceInfo changes to App frontend](base-streaming-service-info-changes-to-frontend_sequencediagram-org.svg){:target="_blank"} + + +### HSM communication + +TODO + +### IPC notifications + +The Middleware is able to receive IPC notifications from other processes running on the BitBoxBase. +IPC notifications are implemented via a Unix named pipe and notifications are formatted in a JSON based protocol. +Using a named pipe allows simple and dependencyless implementation in other Go, Python or Shell Script based processes running on the BitBoxBase. +One draw back is that a named pipe blocks writes until the content is read. +By default the named pipe is located in `/tmp/middleware-notification.pipe` and system level permissions are required to write in the pipe. + +![middleware ipc notifications](middleware_ipc_notifications.png) + +*Schematic showing multiple scripts writing notifications into the named pipe.* + +The JSON based protocol for the notifications is versioned, includes a notification topic and a can contain a payload. +The payload can contain any valid JSON structure. +Since a write operation to a unix pipe only being atomic (i.e. two writes do not interleave) as long as the amount written is smaller than the `PIPE_BUF`, which is 4096 bytes for Linux, the Middleware drops all notifications bigger than 4095 bytes. + +*Sample notification:* +```JSON +{"version": 1, "topic": "sampletopic", "payload": {"sampleInt":123,"sampleString": "string", "sampleBool": true}} +``` diff --git a/docs/customapps/middleware/middleware-configuration.md b/docs/customapps/middleware/middleware-configuration.md index 4b24e212..206dc2b0 100644 --- a/docs/customapps/middleware/middleware-configuration.md +++ b/docs/customapps/middleware/middleware-configuration.md @@ -7,4 +7,48 @@ nav_order: 200 --- ## Middleware: Configuration -(TODO)0xB10C + +### Command line interface parameters + +Running `bbbmiddleware -h` prints the configuration options for the Middleware on the command line. + +Note: *The argument list below was last updated in December 2019.* + +```console +-bbbcmdscript string + Path to the bbb-cmd.sh script that allows executing system commands (default "/opt/shift/scripts/bbb-cmd.sh") +-bbbconfigscript string + Path to the bbb-config.sh script that allows setting system configuration (default "/opt/shift/scripts/bbb-config.sh") +-bbbsystemctlscript string + Path to the bbb-systemctl.sh script that allows starting and stopping services on the Base (default "/opt/shift/scripts/bbb-systemctl.sh") +-datadir string + Directory where the Middleware persistent data, like for example the noise encryption keys, is stored (default ".base") +-electrsport string + Electrs RPC port (default "51002") +-hsmfirmwarefile string + Location of the signed HSM firmware binary (default "/opt/shift/hsm/firmware-bitboxbase.signed.bin") +-hsmserialport string + Serial port used to communicate with the HSM (default "/dev/ttyS0") +-middlewareport string + Port the Middleware listens on (default "8845") +-network string + Indicate wether Bitcoin is running on mainnet or testnet (default "testnet") +-notificationNamedPipePath string + Path where the Middleware creates a named pipe to receive notifications from other processes on the BitBoxBase (default "/tmp/middleware-notification.pipe") +-prometheusurl string + URL of the Prometheus server (default "http://localhost:9090") +-redismock + Flag to use the Redis mock for development instead of connecting to a redis server +-redisport string + Port of the Redis server (default "6379") +-updatehsmfirmware + Set to true to force HSM firmware update +-updateinfourl string + URL to query information about Base image updates from (default "https://shiftcrypto.ch/updates/base.json") +``` + +### Changing the Middleware systemd file + +(TODO)Stadicus + +- config in systemd unit (via env variables / config template in /etc/bbbmiddleware) diff --git a/docs/customapps/middleware/middleware-functionality.md b/docs/customapps/middleware/middleware-functionality.md index 6e150a22..2b13ea2f 100644 --- a/docs/customapps/middleware/middleware-functionality.md +++ b/docs/customapps/middleware/middleware-functionality.md @@ -7,4 +7,45 @@ nav_order: 100 --- ## Middleware: Functionality -(TODO)0xB10C + +The Middleware connects the BitBoxBase system with the BitBoxBase user interfaces, such as the BitBoxApp and the HSM screen. +The task of the Middleware is to respond to synchronous user request and to asynchronously notify the user about changes on the system. +A action from the user could for example toggle _SSH access_ or requests for information about the Bitcoin blockchain synchronization progress. +A notification from the Middleware to the user could be for example about an available software update or the disk running out of space. + +### Tasks of the Middleware + +The Middleware processes Remote Procedure Calls (RPCs) from the BitBoxApp, keeps a connection to the HSM, queries both Redis and Prometheus servers, executes shell scripts to change system configuration and listens on an inter process communication (IPC) interface for notifications from other processes. + +
+![Middleware overview](middleware_overview.png) +*Schematic showing the Middleware and it's connections.* +
+
+ +#### App connectivity via RPC +The BitBoxApp acts as RPC Client talking to the Middleware's RPC Server. +Available RPCs can for example change system configuration, set the user password, create or restore a backup, update the BitBoxBase software or deliver information on services such as Bitcoin Core or c-lightning running on the Base. +The [BitBoxApp <-> Middleware communication section](middleware-communication.html) includes more detail on design and implementation decisions. + +#### HSM connectivity via UART + +The Middleware communicates with the HSM over UART. +Messages are serialized as Protobuf messages. + +#### Querying Redis and Prometheus + +The Redis server on the BitBoxBase acts as a store for system configuration and the Prometheus server holds service specific information. +The Middleware queries information from both. + + +#### Executing shell scripts + +The Middleware calls the [bbb-cmd.sh](../bbb-cmd.html) and [bbb-config.sh](../bbb-config.html) scripts to change system configuration and to call standard commands. + + +#### Listening for IPC notifications + +Processes running on the BitBoxBase can notify the Middleware about changes the user needs to be made aware of. +This is for example used to notify the Middleware about a successful completion of system checks after an update. +The [communication section](middleware-communication.html) includes detailed information about the IPC implementation. diff --git a/docs/customapps/middleware/middleware_ipc_notifications.png b/docs/customapps/middleware/middleware_ipc_notifications.png new file mode 100644 index 00000000..cefd8526 Binary files /dev/null and b/docs/customapps/middleware/middleware_ipc_notifications.png differ diff --git a/docs/customapps/middleware/middleware_overview.png b/docs/customapps/middleware/middleware_overview.png new file mode 100644 index 00000000..5081f9c1 Binary files /dev/null and b/docs/customapps/middleware/middleware_overview.png differ diff --git a/docs/sources/base-pre-update_sequencediagram-org.txt b/docs/sources/base-pre-update_sequencediagram-org.txt new file mode 100644 index 00000000..df43c9ed --- /dev/null +++ b/docs/sources/base-pre-update_sequencediagram-org.txt @@ -0,0 +1,15 @@ +title pre-update + +participant app + +middleware->shiftcrypto.ch/updates/base.json: HTTP GET +middleware<-shiftcrypto.ch/updates/base.json: response +box over middleware: parse JSON +box over middleware: compare base\nversion with \nbase.json version + + +box over middleware: if newer +middleware -->> app: notify app about newer version avaliable + +middleware <- app: GetUpdateVersion +middleware -> app: {\nupdateAvaliable: bool, \nversion: string, \ndescription: string, \nseverity: int\n} diff --git a/docs/sources/base-streaming-service-info-changes-to-frontend_sequencediagram-org.txt b/docs/sources/base-streaming-service-info-changes-to-frontend_sequencediagram-org.txt new file mode 100644 index 00000000..c5f6425d --- /dev/null +++ b/docs/sources/base-streaming-service-info-changes-to-frontend_sequencediagram-org.txt @@ -0,0 +1,33 @@ + +title Streaming ServiceInfo changes to the Frontend + +participant Frontend +participant App +participant Middleware +participant Prometheus +participant Prometheus Scraper + + +Middleware->Prometheus: query last scrape timestamp +Middleware<-Prometheus: timestamp: 2 + +note over Middleware: compare with saved timestamp\n\nequal:\ndo nothing +note over Middleware: wait 5 seconds + +Prometheus <<-- Prometheus Scraper: write new data\ntimestamp: 3 +Middleware->Prometheus: query last scrape timestamp +Middleware<-Prometheus: timestamp: 3 +note over Middleware: compare with saved timestamp\n\nnot equal:\n1. safe timestamp = 3\n2. check if ServiceInfo changed +Middleware->Prometheus: query ServiceInfo data\n(multiple querys) +Middleware<-Prometheus: data ServiceInfo +note over Middleware: check if ServiceInfo data changed\n\n- not changed: skip\n- changed: fire event +App<<--Middleware:notify ServiceInfo changed +note over Middleware:wait 5 seconds +Frontend<<--App:notify ServiceInfo changed + +Frontend->App: GET /service-info +Prometheus <<-- Prometheus Scraper: write new data\ntimestamp: 4 +App->Middleware:GetServiceInfo RPC +App<-Middleware:Return cached\nGetServiceInfoResponse +Frontend<-App:GetServiceInfoResponse +note over Frontend:Redraw diff --git a/docs/sources/base-update-progress-via-rpc_sequencediagram-org.txt b/docs/sources/base-update-progress-via-rpc_sequencediagram-org.txt new file mode 100644 index 00000000..2cccbf33 --- /dev/null +++ b/docs/sources/base-update-progress-via-rpc_sequencediagram-org.txt @@ -0,0 +1,39 @@ +title Base Update Process via RPC + +note over front-end: user wants to update +front-end->App: POST /update-base +App->Middleware: UpdateBase RPC +Middleware->bbb-cmd.sh: mender-update install + +activate bbb-cmd.sh + +bbb-cmd.sh -->> Middleware: read from stdin: progress 1% +Middleware -->> App: Event: UpdateProgresChanged +App -->> front-end: Event: UpdateProgressChanged +note over front-end:Am I ready to query new\nupdate progress info?\n\nNo (do nothing). + +bbb-cmd.sh -->> Middleware: read from stdin: progress 2% +Middleware -->> App: Event: UpdateProgresChanged +App -->> front-end: Event: UpdateProgressChanged +note over front-end:Am I ready to query new\nupdate progress info?\n\nYes. + + +front-end->App: GET /base-update-progress +bbb-cmd.sh -->> Middleware: read from stdin: progress 3% + +App->Middleware:GetBaseUpdateProgress +Middleware->App: 3% +App->front-end: 3% + + +==when update finished== + +bbb-cmd.sh -> Middleware: (no error) +deactivate bbb-cmd.sh + +note over Middleware: set redis keys and restart base + +Middleware -> App: ErrorResponse{Success: true} + +App ->front-end: {success: true} + diff --git a/docs/sources/middleware_ipc_notifications.drawio b/docs/sources/middleware_ipc_notifications.drawio new file mode 100644 index 00000000..f36b82ce --- /dev/null +++ b/docs/sources/middleware_ipc_notifications.drawio @@ -0,0 +1 @@ +3Vldc5s4FP01ntk+2MM3+DFfTdvZzmTXndlt3xQQoFZIrBDB3l/fKxAGDHHj1EntOjMxOpIu4tx7DkbM7KtsfStQnn7kEaYzy4jWM/t6Zlmm43vwpZCNRowgaJBEkEhjHbAi/+N2oEZLEuFiMFByTiXJh2DIGcOhHGBICF4Nh8WcDs+aowSPgFWI6Bj9h0QybdDANTr8HSZJ2p7ZNHRPhtrBGihSFPGqB9k3M/tKcC6bo2x9halir+Wlmff2kd7twgRm8ikT0H8F/7t6/+V9tl46fyUfPhkfbudOE+UB0VJf8EcSRRRXEBZwgVFUwDfjksQkRJJwVujLkZuWIxwBZbrJhUx5whmiNx16KXjJIqwWYkCrG/Mn5zmAJoBfsZQbnX9USg5QKjOqe/GayH/V9IWrW597PddrHblubNoGk2LTm6San/t93bS6NZh3hwXJsMRCgzFnUq/OVNFiQukVp1zUFNhxjL0wBLyQgn/DvZ7IX94bKgLkP1fTY4rXF6oyAat0SZnNedv6dxaeD8A4wzrpBS9FiPektVUKEgmWe8bZzTiVwN4JdP3cYg4MiE1dBxRy/zDUBNLSSrbj9FS4OrTpDcg5YbLoRb5TAAzQNjF3bK0R7RKWu1PKB06Ag2YNbat3MR1U6+MArZh7tbKjia7iVZFWKZF4laM6aRVY5bC6d4vr0cQ/YCHxem+qthQthxSZnm5XnY2ZrTelPQtrx02lt0fw4fzZI/4gjMxyFWxL5LxvNYuc5GNqWyGFSES/hls32Cm/5Zhb25jg1n8pbq3lM1x5ypGO5tRmz6c71/6hU1uG3fdq8yW82lV/U17t1Z+fcl73VznvT5WPO5JmOKeqbBlhiWKClglhM8ujcE2X9wKOEnX0x90GCoS9Oa797eQLm5GL/al8LT3fRnvzdYCofWMoasecMEz7NQ3TOzVNH/Lr68m+ezRJuSclKcs5uew9z5G9M3fk4CwdORg5cs4LOS/zCEk8L0JBcjnhx6sUYr+sHYMZB5EzlazAurf3J+sAO7b9oR27xoQdO69px2Zwaop+5m8sxxwo2lgsA+/cVN3enM9M1u2ye7pelTkWD6TgYkLPt5wilrywoBEO4sm9Cy8M8H18JEF71hME/aq/r6yTE/Qzb9GB6Z+/oM/zyckcPzrdCRU7xWUxFjTcuKF8ppT+Gz1JWcETnqRe9dZt+aMsZaV6iQAsj1KRCx7iosBq47sSwPd4A9wAAcO/tO5CGaQHOJvaqwLK5DBFI+qN+jOhtvoDOIIncQZYCClRUr1UqYC10Avd0eyePVIdQ2/aLZDtSwnVmQgUEThJuwjGmYqaty6hIqv9APv6WJXiLP2F++Nasd1xrQSH1wo0u9ctza5w99bKvvkO \ No newline at end of file diff --git a/docs/sources/middleware_overview.drawio b/docs/sources/middleware_overview.drawio new file mode 100644 index 00000000..3ecb1e9f --- /dev/null +++ b/docs/sources/middleware_overview.drawio @@ -0,0 +1 @@ +7Vpbc+MmFP41ntk+JCMJSY4fYyfZZGYzk6mnbbZvRMIWDRZehG/76wsCdLfjxJLtdJsHRxzgCM75zg3UA6PZ+iuD8+iRhoj0HCtc98BNz3H8gSt+JWGjCe6VIkwZDhXJzglj/BNpoqWpCxyipDSQU0o4npeJAY1jFPASDTJGV+VhE0rKb53DKaoRxgEkdepfOOSRol45/Zx+j/A0Mm+2/YHqmUEzWO8kiWBIVwUSuO2BEaOUq6fZeoSIlJ2Ri5p3t6U3WxhDMd9nAv7zgSxHq2/Pfw8ff8A/oHVzPbqwHcVmCclC71ivlm+MCBhdxCGSXKweGK4izNF4DgPZuxI6F7SIz4ho2eJxQmN+B2eYSH3fI7JEHAdQd2j12gPRri9f72iJGEfrAklv5yuiM8TZRgzRvY6vRauxdaWbq1xRrqdpUUFJjiFCDY5pxjqXn3jQInyPOL2a9FAo8KSblPGITmkMyW1OHZblm4/5RulcS/UfxPlGSw8uOC3LHK0xf5bTLz3d+q6ZyeebdbGxMY1Y7Pe52Piec5DNfFraMvMSDhm/loYlCAGBSYIDQ77D5N0wQHFouMU0RoqiGVm7YJLQBQvQLl1obwHZFPEd43w1TuppJ+gYIpDjZdkvNCEonSo2BTeFAXOKY54UOD9JQo5lUMGy7VoVNCqOOTazpR0AV/8kcD0X6DWajb3TbLqHow06wWMNcK5TBpzwimUWakd6Vo7EQ4FtvHSnwHbAKf2wXYBTDq6T+OGCe80H5R52b3sBkncE57JBcPx6HFPot20KBwV3Z3A+oHL+R9VHUeW4ZxHwPa8S8AdeOX1/a/xREoSG6sAnXKuxZAv+jwU1HRdJquBrMcAG83WqVtMvnqby/yMOQ4JWQtGGpVih4qoGHKEOAe3UIcCqqKahELGdhkLE76oOOUkZspfIXdGWkhU95JrgaSyIsxQKkhll+KcYDA3LPf3Rx52Gu28oOqtI5G41y5e9bHIgbNJR6ayV2tFFogxJ9saUzYQCagY7xHxI1wVjfcktVdEEk7i791/P54WXq3cdzVO47XgK++w8hb8VSiFe7u3ga8p6NyQauTzEHDHR+8RogBIRTK0Rnc0WsdAPxzTOQM8+/oovD0+j37biKiOnwvhsccmtos0+NdrMAg7yXE3ZxHbnNIQJqqlMCJCX9ZJwRl/RiBLKetlR1ESEmAoJ6pgVoBSbO4JZExDKUDnqWWm/jIVBAxTsprPSzqAAtkJBO4003pfQkPCi5g9IPu8hC1XqaY1RsGCYS1E90nBBJO3L/fhxu1P4rOafHXufzvz7NZ2PIzVvHDA8V/XS55Cu1z+3UO40VWvVMqCpcv+IBJuPJWy/v/tcQjSeEMNiw9J57j5SfTPZLki66ZrH0A48KM3cYnaAObj0nDKXLWeldV61w1DBq8xKlSI1Vm1V9KBugiKfCjFDgUypZOL9Zqw0MZCgCd8VARkSrhi+pIykpvWRiODqDXvejdQ9GBL4gsgQBq/T1NRNuO05YJL+VbC3M27WobTbUKrGnl0j60X3ije1TU7AunRc128FZuZ0cWM4V3BBJ5MEdQIJ4ybe7Te2VOetOuRzdQJ+zXA/7APqrLr2AYOawl9+NRdgMN+CC3DdAWjHBYAjGv32M6SDCn9RqoulRWiRfPbquZbgnbx6Nh8dnejrkt7+t5rvv0dq/Phj/2zU3Hdl3xeoVfbfvPBq6/bJ3J4f/yS5ldunyq382+MdrwLnLm7xvW581O8oxP8593T66h40qaur+nNfwz3T/BFYreWPDay6zh/rlwY/Fkhu/w6tUfALpI8Z1NuoIH0AypbcTjJ5ZVBRZntIaima+ffKanj+0Te4/Rc= \ No newline at end of file diff --git a/docs/update/base-pre-update_sequencediagram-org.svg b/docs/update/base-pre-update_sequencediagram-org.svg new file mode 100644 index 00000000..76e74c25 --- /dev/null +++ b/docs/update/base-pre-update_sequencediagram-org.svg @@ -0,0 +1 @@ +pre-updateappmiddlewareshiftcrypto.ch/updates/base.jsonHTTP GETresponseparse JSONcompare baseversion with base.json versionif newernotify app about newer version avaliableGetUpdateVersion {updateAvaliable: bool, version: string, description: string, severity: int} \ No newline at end of file diff --git a/docs/update/base-update-progress-via-rpc_sequencediagram-org.svg b/docs/update/base-update-progress-via-rpc_sequencediagram-org.svg new file mode 100644 index 00000000..b3e0a255 --- /dev/null +++ b/docs/update/base-update-progress-via-rpc_sequencediagram-org.svg @@ -0,0 +1 @@ +Base Update Process via RPCfront-endAppMiddlewarebbb-cmd.shuser wants to update POST /update-base <version>UpdateBase RPC <version>mender-update install <version>read from stdin: progress 1%Event: UpdateProgresChangedEvent: UpdateProgressChangedAm I ready to query newupdate progress info?No (do nothing).read from stdin: progress 2%Event: UpdateProgresChangedEvent: UpdateProgressChangedAm I ready to query newupdate progress info?Yes.GET /base-update-progressread from stdin: progress 3%GetBaseUpdateProgress3%3%when update finished(no error)set redis keys and restart base ErrorResponse{Success: true}{success: true} \ No newline at end of file diff --git a/docs/update/update-notification.md b/docs/update/update-notification.md index 7febf6ee..9028dac2 100644 --- a/docs/update/update-notification.md +++ b/docs/update/update-notification.md @@ -8,4 +8,6 @@ nav_order: 150 When the BitBoxApp is connected to the BitBoxBase, the version of the current Base image (and of Bitcoin-related services) is visible in the Node Management dashboard. -If an update is available, the App will prompt with a brief release information. +To check if a newer Base image is available, the Middleware checks [`shiftcrypto.ch/updates/base.json`](shiftcrypto.ch/updates/base.json) and compares it with the local semantic version. If a newer version is available, the App is notified and the user can trigger the system update. + +![Update trigger](base-pre-update_sequencediagram-org.svg) diff --git a/docs/update/update-process.md b/docs/update/update-process.md index 5fa2f22d..76d42695 100644 --- a/docs/update/update-process.md +++ b/docs/update/update-process.md @@ -20,6 +20,8 @@ This post-processing creates Mender update artifacts for over-the-air (OTA) upda The disk image has the Mender client built in. It is executed on demand by the Middleware to update the device. +![Mender update process](update_mender_process.png) + On the BitBoxBase, only one root filesystem - running the operating system and applications - is active at any given time, e.g. "root filesystem A". When the update process is started, the new disk image is streamed directly to the non-active root filesystem, e.g. "root filesystem B". After successful verification of the update signature and completion of the download, the bootloader is configured to boot from the updated "root filesystem B" once. @@ -31,6 +33,10 @@ If everything works as expected, the updated "root filesystem B" is commited to If the device is unable to boot, or if application-level checks fail after the update, the system automatically falls back to booting to the previous "root filesystem A", which was working before the update. This process makes it very unlikely that a device will end up in a state that can't be fixed using further OTA update. -![Mender update process](update_mender_process.png) - See additional information on + +## RPC communication + +The update process involves the BitBoxApp communicating with the Base Middlware over RPCs. + +![Update RPC communication diagram](base-update-progress-via-rpc_sequencediagram-org.svg) diff --git a/docs/update/update_mender_process.png b/docs/update/update_mender_process.png index 1e2ba229..2632af46 100644 Binary files a/docs/update/update_mender_process.png and b/docs/update/update_mender_process.png differ