Copying files from Lighthouse to Nodes with node-copy

The node-copy CLI tool allows Lighthouse Administrators to easily copy files or directories from Lighthouse to a Managed Node or in parallel on many or all Nodes.

Getting Started

To get started with any of the node-* tools, you can get quick information on how to use it from the CLI:

sudo node-copy --help
usage: node-command [options] command
Common options:
-h --help Display this message
-q --quiet Suppress command output
-b --batch Suppress node-command output
-l --list-nodes List all nodes matching query, or all nodes if none selected
-i --node-id=ID Select node by config ID
-n --node-name=name Select node by name
-a --node-address=address Select node by VPN address
-g --smartgroup=name Select nodes by the smart group they resolve to
-A --all Select all available nodes
-C --connected Select all connected nodes
-P --pending Select all pending nodes
-D --disconnected Select all disconnected nodes
-d --disable-fslog Disable logging to file system
-p --par=num Number of parallel processes to run at a time
-c --copy-file Copy a file to the node(s).
-s --source-file The file to copy to the node(s).
Required if --copy-file is
specified.

To see a list of all the enrolled Nodes that the tool can operate on:

sudo node-info --all

This command displays the following information:

<Node's name in Lighthouse>
	address: <Node's Lighthouse VPN address>
	id: <Node's unique identifier in Lighthouse>
	ssh port: <Node's SSH server port>
	description: <Optional description of node in Lighthouse>
	enrollment status: <Whether a node is "Registered" pending enrollment, or "Enrolled">
	connection status: <Whether a node is currently "Connected" or "Disconnected" from Lighthouse VPN>

Selecting Nodes

There are a number of ways to select Nodes as targets on which to run a command, listed below. These can be used multiple times, or together, to select multiple Nodes.

Select individually by name, Lighthouse VPN address, or config ID, as per node-info output:

sudo node-copy --node-name BNE-R01-IM4248
sudo node-copy --node-address 192.168.128.50
sudo node-copy --node-index nodes-49
sudo node-copy --node-index nodes-1 --node-index nodes-2 --node-index nodes-3

Select all:

sudo node-copy --all

Select a subset of nodes, using a pre-configured Smart Group:

sudo node-copy --smartgroup "Brisbane Nodes"

Copying Files/Folders 

Once Nodes have been selected, You can copy desired files/folders directly to the nodes from the Lighthouse CLI.

For example, you can copy a backup file directly to the node:

sudo node-copy -i nodes-34 -s /tmp/backup-file.export -c /tmp

Note: When running a copy on many or all Nodes, we recommend testing the copy on one or just a few Nodes first.

Output Format: CLI

The command outputs the result of each copy run on each Node:

sudo node-copy -i nodes-34 -s /tmp/backup-file.export -c /tmp
Copying '/tmp/backup-file.export' -> '/tmp'
Removing oldest log: /var/log/node-command/2022-08-28_13933
== node-command ID 2023-06-27_23524 ==
04:10:22 [SUCCESS] om2248-l /var/log/node-command/2023-06-27_23524/stdout/nodes-34

There are a few components to this output:

1. The first line displays the run ID. This is the unique ID for this command including a timestamp of when the command was run, used to locate historical logs (discussed below).
2. Each node copy result has a result header line. This contains the time the copy was completed, if the copy succeeded or failed, the node name, and the node address.

There are a few ways to modify the output of the command, useful for batch operation. To hide the command output results, use the --quiet argument and only the headers will be shown. To suppress headers and display command output only, use the --batch argument. Combine both arguments to hide all output.

Output Format: Logs

Information about each run is logged to the filesystem by default. Filesystem logging can be disabled by using the --disable-fslog argument. The logs are stored in /var/log/node-command/, and are indexed using the run ID of each command (as detailed above in the Output Format: CLI section).

A new directory is created for each run, and contains 3 items:

1. A targets.txt file, listing the addresses on which the command was run.
2. A stdout directory, containing the output to stdout for the command printed by each Node.
3. A stderr directory, containing the output to stderr for the command printed by each Node.

By default, a history of the last 30 commands are kept logged to the file system before being removed. Example of our latest copy and all the history recorded:

ls -laR /var/log/node-command/2023-06-27_23524/
/var/log/node-command/2023-06-27_23524/:
total 20
drwxr-x--- 4 root root 4096 Jun 27 04:10 .
drwxr-x--- 32 root root 4096 Jun 27 04:10 ..
drwxr-x--- 2 root root 4096 Jun 27 04:10 stderr
drwxr-x--- 2 root root 4096 Jun 27 04:10 stdout
-rw-r----- 1 root root 17 Jun 27 04:10 targets.txt

/var/log/node-command/2023-06-27_23524/stderr:
total 8
drwxr-x--- 2 root root 4096 Jun 27 04:10 .
drwxr-x--- 4 root root 4096 Jun 27 04:10 ..
-rw-r----- 1 root root 0 Jun 27 04:10 192.168.128.5:22
lrwxrwxrwx 1 root root 16 Jun 27 04:10 nodes-34 -> 192.168.128.5:22

/var/log/node-command/2023-06-27_23524/stdout:
total 8
drwxr-x--- 2 root root 4096 Jun 27 04:10 .
drwxr-x--- 4 root root 4096 Jun 27 04:10 ..
-rw-r----- 1 root root 0 Jun 27 04:10 192.168.128.5:22
lrwxrwxrwx 1 root root 16 Jun 27 04:10 nodes-34 -> 192.168.128.5:22

Syslog Information

In addition to the output and file logging, the running of commands is also recorded in syslog (/var/log/messages from the CLI).

2023-06-27T04:18:46.504945+00:00 lighthouse node-command[9829]: User 'root' copied file '/tmp/backup-file.export' to '192.168.128.5:22:/tmp'
2023-06-27T04:18:46.518904+00:00 lighthouse cli_log: [root] - node-copy -i nodes-34 -s /tmp/backup-file.export -c /tmp