Security: Configuration Restrictions

[aiwaniuk]

https://github.com/open-telemetry/opamp-spec/blob/main/specification.md#configuration-restrictions

This is very hard thing to implement in a secure way, but if not implemented properly then it may even decrease security of environment: Agent operator assuming it works correctly, can possibly ignore other protection layers (like not running as root/user with access to important files).

Doing it on config validation level introduces some risks:

• validation need to properly handle dir traversal cases, like: "log_file: /allowed_dir/../etc/passwd"
• validation need to handle symlinks (for example log_file: /allowed_dir/symlink_file, symlink_file -> /etc/passwd) and if this is done only on config apply level then symlink_file location may change latter
• also many Agents probably will be build using some collection plugins with different configuration schemes and writing generic validation will be hard (it need to be written to each plugin separately)
• also some plugins may not even have path to files configurable, but hardcoded in executable, then it will require knowledge that some plugin is accessing some file

Doing it on runtime:
This can be done by implementing some common API to work with filesystem and apply all validation on that level. But this is also error prone, agent developers need to somehow enforce that regular API for accessing files is not used.
Similar feature was present in PHP (https://www.php.net/manual/en/ini.core.php#ini.open-basedir, https://externals.io/message/105606) and it was a lot of pain from security point of view.

Restrictions on accessing system API:
This is something that may work. Sandboxing agent process and perform checks when executables are accessing system API (for interaction with filesystem/executing executables).

Other things that it is worth to restrict next to accessing filesystem and executing commands:
SSRF bugs: If something supports some collection from url (or process some urls from configs), it may be also worth to restrict these somehow, to avoid situation when send configuration will have link for example to AWS metadata (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html): http://169.254.169.254/latest/meta-data/iam/security-credentials/s3access which contains AWS creds for role assigned with instance

If we want to include recommendation for implementing restrictions on directories/executables sandbox approach should be recommended instead of config file validation. I believe this will allow to avoid a lot of pain from Agent implementers who will decide to implement configuration restrictions.

[tigrannajaryan]

@aiwaniuk good comments. The configuration is just one of the protection layers and is a weak one if we are honest and like you said very hard to implement in presence of pluggable collection components. It is probably only adequate for simple agents where the entire codebase is tightly controlled and can be guaranteed to follow whatever restrictions are put in the configuration.

I would definitely want to know how we can do better and how for example the sandboxing can work.

[aiwaniuk]

Sandboxing:
General idea here is to do some filtering on system calls done by process and ensure these are not accessing files not allowlisted. Since this is done on system calls level, all code in agent plugins will go through this path.
In Linux system there are few methods that may allow doing this:

• secomp/eBPF
• AppArmor
• namespaces (process may be setup in a way that only allowlisted locations are visible (mounted) in filesystem)

some examples that implements such things:

• containers
• nsjail: https://github.com/google/nsjail

some addiational resources: https://developers.google.com/code-sandboxing

I'm not familiar with other systems like Windows.

[tigrannajaryan]

Thanks @aiwaniuk

I think for something complex like that we would want to see an experimental implementation to help us understand how exactly we should modify the spec to adopt it. Ideally we would want to see a working implementation in opamp-go Agent example.

I already added "help wanted" label so if anyone wants to give it a try it is available.