Compare commits
16 commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
699ed7be40 | ||
|
|
01625655e7 | ||
|
|
82d8fa7509 | ||
|
|
12084372a8 | ||
|
|
3c1df01014 | ||
|
|
e1cef70830 | ||
|
|
8f358517e5 | ||
|
|
480d4f7e3d | ||
| 2e40d2061a | |||
| b54f6811f3 | |||
| d0f9653b17 | |||
| adbf30f77f | |||
| e7e9332397 | |||
| 59516c37db | |||
|
|
8dba25c31f | ||
| 592f8144fc |
12 changed files with 220 additions and 0 deletions
9
.gitignore
vendored
Normal file
9
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,9 @@
|
||||||
|
|
||||||
|
.vs/
|
||||||
|
.vscode/
|
||||||
|
.idea/
|
||||||
|
.env
|
||||||
|
test.db
|
||||||
|
|
||||||
|
*.pdf
|
||||||
|
target/
|
||||||
5
.vscode/settings.json
vendored
Normal file
5
.vscode/settings.json
vendored
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
{
|
||||||
|
"editor.tabSize": 2,
|
||||||
|
"editor.detectIndentation": false,
|
||||||
|
"editor.insertSpaces": false
|
||||||
|
}
|
||||||
21
assets/Design.svg
Normal file
21
assets/Design.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 88 KiB |
14
chapters/01-Project-Requirements.typ
Normal file
14
chapters/01-Project-Requirements.typ
Normal file
|
|
@ -0,0 +1,14 @@
|
||||||
|
|
||||||
|
= Project Requirements
|
||||||
|
|
||||||
|
This chapter will outline the requirements
|
||||||
|
to be fulfilled by this project in regards to the DHBW.
|
||||||
|
|
||||||
|
Eines der Gruppenmitglieder muss bis zum TODO folgende Dateien hier hochladen:
|
||||||
|
|
||||||
|
- Ausarbeitung als PDF (use dhbw template later on)
|
||||||
|
- Präsentation als PDF
|
||||||
|
- Textdatei mit Teilnehmern, Projekttitel und Kurzbeschreibung
|
||||||
|
- Programmentwurf (komplettes Repository, was bewertet wird) als ZIP / TAR
|
||||||
|
|
||||||
|
Die Prüfungsleistung muss bis spätestens zum TODO hier abgegeben werden. Spätere Abgaben sind nicht möglich.
|
||||||
5
chapters/02-Project-Goal.typ
Normal file
5
chapters/02-Project-Goal.typ
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
|
||||||
|
= Project Goal
|
||||||
|
The aim of this project is to reimplement a subset of the functionality of the secret management tool "#link("https://github.com/hashicorp/vault")[HashiCorp Vault]", which is currently written in Go, in Rust.
|
||||||
|
Expected benefits of this are an increase in speed and reliability and a higher level of security.
|
||||||
|
As part of this, a HTTP-API for key-value based secret handling is implemented. Data is stored in a sqlite database.
|
||||||
2
chapters/03-Planning-and-Timeline.typ
Normal file
2
chapters/03-Planning-and-Timeline.typ
Normal file
|
|
@ -0,0 +1,2 @@
|
||||||
|
|
||||||
|
= Planning and Timeline
|
||||||
5
chapters/04-Architecture.typ
Normal file
5
chapters/04-Architecture.typ
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
|
||||||
|
= Architecture
|
||||||
|
|
||||||
|
#include "./architecture/System-Design.typ"
|
||||||
|
#include "./architecture/Dynamic-Routing.typ"
|
||||||
14
chapters/05-API-Compliance-and-Testing.typ
Normal file
14
chapters/05-API-Compliance-and-Testing.typ
Normal file
|
|
@ -0,0 +1,14 @@
|
||||||
|
|
||||||
|
= API-Compliance and Testing
|
||||||
|
|
||||||
|
One of the goals of this implementation is to provide compatability to the current version of the vault written in Go.
|
||||||
|
Therefore, the #link("https://github.com/hashicorp/vault-client-go")[HashiCorp Go-client] is used for testing.
|
||||||
|
|
||||||
|
To ensure compatability, the #link("https://github.com/hashicorp/vault-client-go/blob/main/openapi.json")[OpenAPI specification] and the #link("https://developer.hashicorp.com/vault")[Vault documentation], both published by HashiCorp, are used as references. They provide information on input parameters, types and return values.
|
||||||
|
|
||||||
|
Implemented test cases are:
|
||||||
|
- Write a secret
|
||||||
|
- Destroy specific versions of a secret
|
||||||
|
- Read a secret
|
||||||
|
- Read a metadata entry
|
||||||
|
- Write a metadata entry
|
||||||
67
chapters/architecture/Dynamic-Routing.typ
Normal file
67
chapters/architecture/Dynamic-Routing.typ
Normal file
|
|
@ -0,0 +1,67 @@
|
||||||
|
|
||||||
|
== Contextual Routing
|
||||||
|
|
||||||
|
This issue has previously been adressed as "dynamic routing".
|
||||||
|
|
||||||
|
// Problem can be split in two, // TODO
|
||||||
|
|
||||||
|
In Axum, routers can be configured using a fluid-like API #footnote[Term borrowed from C\# and OOP].
|
||||||
|
|
||||||
|
#box[```rs
|
||||||
|
#[tokio::main]
|
||||||
|
async fn main() {
|
||||||
|
let app = Router::new()
|
||||||
|
.route("/", get(root))
|
||||||
|
// .route("/posts", get(get_posts))
|
||||||
|
.route("/posts/:post_id", get(get_post_by_id))
|
||||||
|
.route("/posts/:post_id/discussions/+thread_ids", get(get_posts_threads));
|
||||||
|
|
||||||
|
todo!("Actually listen")
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn root() {
|
||||||
|
todo!()
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn get_post_by_id(Path(post_id): Path<String>) -> impl IntoResponse {
|
||||||
|
todo!()
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn get_posts_threads(Path(post_id): Path<String>, Path(discussion_thread_ids): Path<String>) -> impl IntoResponse {
|
||||||
|
let discussion_thread_ids = discussion_thread_ids.split("/");
|
||||||
|
|
||||||
|
todo!()
|
||||||
|
}
|
||||||
|
```]
|
||||||
|
|
||||||
|
As it can be seen, a parameter can be extracted from the URL path by using `/:some_name` and can be placed as any infix of the URL: at the beginning, in the middle, or the end.
|
||||||
|
However, as the path is split by slash characters (`/`), the extracted part must not contain slashes and hence can only be (up to) one part of the path.
|
||||||
|
|
||||||
|
Longer parts of the request can be obtained using `+some_name` and can only be placed as a postfix of the URL, but not other types of infixes.
|
||||||
|
This can be used to obtain the remaining path of the request.
|
||||||
|
|
||||||
|
=== Problem Description
|
||||||
|
|
||||||
|
+ The API to implement e.g. has the concept of mount points (similar to how filesystems can be mounted on UNIX-like systems).
|
||||||
|
Mount points can contain multiple slashes.
|
||||||
|
For example `/v1/some/mount/point/data/some/path/secret` may consist of a mount point `some/mount/point` and further, following routes of the mapped secret engine. //TODO
|
||||||
|
In this example, `/data` is related to the Key-Value engine and `/some/path/secret` specifies a path within the secret engine instance mounted at the mount point.
|
||||||
|
This implies a significant problem:
|
||||||
|
How to determinate what part of the URL displays a mount point, where it is not certan, what the postfix of the URL will be (this problem follows).
|
||||||
|
|
||||||
|
+ Another problem is that based on the mount point, the request must be processed by the applicable secret engine.
|
||||||
|
Based on context, stored on the DBMS, the request must be passed to the secret engine along with the determinated mount point.
|
||||||
|
|
||||||
|
=== Solution
|
||||||
|
|
||||||
|
Secret Engines have their dedicated router.
|
||||||
|
The main router has an instance of these routers along with the database pool wrapped within a struct in its state.
|
||||||
|
The router instances have a reference to the database pool, which is internally wrapped by an Atomic Reference Counter (`Arc`).
|
||||||
|
|
||||||
|
Upon a requst, the remaining path is obtained (via `/+mount_path`).
|
||||||
|
Then, the path is looked up at the database, also requsting the engine type.
|
||||||
|
If not found, the last last slash character and the following string is removed and looked up again. This is repeated, until the path either is found or has a length of zero, rejecting the request as "404 Not Found".
|
||||||
|
|
||||||
|
If found, the router is called with the request and mount path is given to the router as an "Extension".
|
||||||
|
The `call` #link("https://docs.rs/tower/0.4.13/tower/trait.Service.html#tymethod.call")[(link)]
|
||||||
|
function which Axum routers inherit from the Tower crate, allows to hand the request over to engine's router.
|
||||||
35
chapters/architecture/System-Design.typ
Normal file
35
chapters/architecture/System-Design.typ
Normal file
|
|
@ -0,0 +1,35 @@
|
||||||
|
== System Design
|
||||||
|
|
||||||
|
=== Clients
|
||||||
|
|
||||||
|
The rvault server is compliant with any client acting in compliance with the hashicorp vault api specification.
|
||||||
|
To achieve this, tests are written using the official Hashicorp vault go client.
|
||||||
|
|
||||||
|
=== Webserver
|
||||||
|
|
||||||
|
The Webserver forwards client requests to different routers according to the request paths:
|
||||||
|
i.e. "/v1/auth" for the authorization router.
|
||||||
|
Those routers map requests to their corresponding handlers.
|
||||||
|
|
||||||
|
Problem:
|
||||||
|
- TODO describe middleware
|
||||||
|
|
||||||
|
=== Engines
|
||||||
|
|
||||||
|
Engines are used to manage data. Depending on the engine this can include storage, generation and encryption. Each engine is independent from other engines.
|
||||||
|
Different engines are addressed via a path, which is passed as a prefix to the secret path. The path for the key-value storage would be .../kv-v2/foo.
|
||||||
|
|
||||||
|
In the code, engines are represented by folders which are divided into a logic section, struct sections and a test section.
|
||||||
|
|
||||||
|
=== Storage
|
||||||
|
|
||||||
|
Data is currently stored in an SQLite databse which is accessed with SQLX without ORM. Later stages of the project will allow for different storage systems to be used.
|
||||||
|
|
||||||
|
=== Overview
|
||||||
|
|
||||||
|
#figure(
|
||||||
|
image("../../assets/Design.svg", width: 80%),
|
||||||
|
caption: [
|
||||||
|
The acting components of rvault.
|
||||||
|
],
|
||||||
|
)
|
||||||
22
documentation.typ
Normal file
22
documentation.typ
Normal file
|
|
@ -0,0 +1,22 @@
|
||||||
|
#import "@preview/arkheion:0.1.0": arkheion, arkheion-appendices
|
||||||
|
|
||||||
|
#show: arkheion.with(
|
||||||
|
title: [Secret Management with rvault],
|
||||||
|
authors: (
|
||||||
|
(name: "someone", email: "", affiliation: "HPE"),
|
||||||
|
(name: "someone", email: "", affiliation: "HPE"),
|
||||||
|
(name: "(someone)", email: "", affiliation: "HPE"),
|
||||||
|
),
|
||||||
|
// Insert your abstract after the colon, wrapped in brackets.
|
||||||
|
// Example: `abstract: [This is my abstract...]`
|
||||||
|
abstract: [The goal of this project is to implement the HashiCorp Vault in Rust. During this project we have developed an architecture and started implementation of some cases for key-value-storage. Further development is needed to achieve feature parity with the original version written in Go.],
|
||||||
|
// keywords: ("First keyword", "Second keyword", "etc."),
|
||||||
|
date: "2024-06-02",
|
||||||
|
)
|
||||||
|
#show link:underline
|
||||||
|
|
||||||
|
//#include "./chapters/01-Project-Requirements.typ"
|
||||||
|
#include "./chapters/02-Project-Goal.typ"
|
||||||
|
//#include "./chapters/03-Planning-and-Timeline.typ"
|
||||||
|
#include "./chapters/04-Architecture.typ"
|
||||||
|
#include "./chapters/05-API-Compliance-and-Testing.typ"
|
||||||
21
rvault_description.txt
Normal file
21
rvault_description.txt
Normal file
|
|
@ -0,0 +1,21 @@
|
||||||
|
Secret Management with rvault
|
||||||
|
|
||||||
|
Groupmembers:
|
||||||
|
[redacted]
|
||||||
|
|
||||||
|
Description:
|
||||||
|
The goal is to rewrite a part of the Hashicorp vault in Rust.
|
||||||
|
It should be compatible with the current version of the vault an its clients.
|
||||||
|
Therefore a vault client written in go is used to perform tests.
|
||||||
|
During this project only a small part will be implemented.
|
||||||
|
It will be further developed as an open-source project.
|
||||||
|
Implemented features include:
|
||||||
|
|
||||||
|
- Design of the architecture
|
||||||
|
- Implementation of dynamic routing to allow for exchangeable secret engines
|
||||||
|
- Basic kv-store
|
||||||
|
|
||||||
|
The main development branch is the engine-kv branch.
|
||||||
|
|
||||||
|
To execute the tests navigate to rvault/go_client/tests and execute "go test -v ./..."
|
||||||
|
To clear the test cache use "go clean -testcache"
|
||||||
Loading…
Reference in a new issue