Compare commits

...

16 commits
dev ... docs

Author SHA1 Message Date
someone
699ed7be40
Added testing instructions 2024-06-02 23:04:53 +02:00
someone
01625655e7
further doc work 2024-06-02 23:00:04 +02:00
sam
82d8fa7509
+ sysdesign v0.5
- new abstract
2024-06-02 13:39:49 -07:00
someone
12084372a8
Further doc work 2024-06-02 22:29:10 +02:00
someone
3c1df01014
Edited Compliance and testing 2024-06-02 21:53:11 +02:00
sam
e1cef70830
+ system design mock 2024-06-02 12:54:06 -07:00
someone
8f358517e5
updated Compliance and testing 2024-06-02 21:11:07 +02:00
someone
480d4f7e3d
Changes to be committed:
modified:   documentation.typ
	new file:   rvault_description.txt
2024-06-01 14:15:21 +02:00
2e40d2061a
Docs (architecture): Describe Contextual Routing 2024-05-08 15:05:11 +02:00
b54f6811f3
Restructure: Split chapters into dedicated files 2024-05-08 15:04:38 +02:00
d0f9653b17
Add .env and test.db to .gitignore 2024-05-07 16:21:48 +02:00
adbf30f77f
Add structure to doc 2024-04-11 00:04:27 +02:00
e7e9332397
fix: gitignore, swith to typst 2024-04-10 17:44:53 +02:00
59516c37db
fix: gitignore, create document root 2024-04-10 17:42:52 +02:00
Samuql
8dba25c31f
Create README.md 2024-04-08 15:10:05 +02:00
592f8144fc
Add gitignore 2024-03-18 15:19:25 +01:00
12 changed files with 220 additions and 0 deletions

9
.gitignore vendored Normal file
View file

@ -0,0 +1,9 @@
.vs/
.vscode/
.idea/
.env
test.db
*.pdf
target/

5
.vscode/settings.json vendored Normal file
View file

@ -0,0 +1,5 @@
{
"editor.tabSize": 2,
"editor.detectIndentation": false,
"editor.insertSpaces": false
}

21
assets/Design.svg Normal file

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 88 KiB

View file

@ -0,0 +1,14 @@
= Project Requirements
This chapter will outline the requirements
to be fulfilled by this project in regards to the DHBW.
Eines der Gruppenmitglieder muss bis zum TODO folgende Dateien hier hochladen:
- Ausarbeitung als PDF (use dhbw template later on)
- Präsentation als PDF
- Textdatei mit Teilnehmern, Projekttitel und Kurzbeschreibung
- Programmentwurf (komplettes Repository, was bewertet wird) als ZIP / TAR
Die Prüfungsleistung muss bis spätestens zum TODO hier abgegeben werden. Spätere Abgaben sind nicht möglich.

View file

@ -0,0 +1,5 @@
= Project Goal
The aim of this project is to reimplement a subset of the functionality of the secret management tool "#link("https://github.com/hashicorp/vault")[HashiCorp Vault]", which is currently written in Go, in Rust.
Expected benefits of this are an increase in speed and reliability and a higher level of security.
As part of this, a HTTP-API for key-value based secret handling is implemented. Data is stored in a sqlite database.

View file

@ -0,0 +1,2 @@
= Planning and Timeline

View file

@ -0,0 +1,5 @@
= Architecture
#include "./architecture/System-Design.typ"
#include "./architecture/Dynamic-Routing.typ"

View file

@ -0,0 +1,14 @@
= API-Compliance and Testing
One of the goals of this implementation is to provide compatability to the current version of the vault written in Go.
Therefore, the #link("https://github.com/hashicorp/vault-client-go")[HashiCorp Go-client] is used for testing.
To ensure compatability, the #link("https://github.com/hashicorp/vault-client-go/blob/main/openapi.json")[OpenAPI specification] and the #link("https://developer.hashicorp.com/vault")[Vault documentation], both published by HashiCorp, are used as references. They provide information on input parameters, types and return values.
Implemented test cases are:
- Write a secret
- Destroy specific versions of a secret
- Read a secret
- Read a metadata entry
- Write a metadata entry

View file

@ -0,0 +1,67 @@
== Contextual Routing
This issue has previously been adressed as "dynamic routing".
// Problem can be split in two, // TODO
In Axum, routers can be configured using a fluid-like API #footnote[Term borrowed from C\# and OOP].
#box[```rs
#[tokio::main]
async fn main() {
let app = Router::new()
.route("/", get(root))
// .route("/posts", get(get_posts))
.route("/posts/:post_id", get(get_post_by_id))
.route("/posts/:post_id/discussions/+thread_ids", get(get_posts_threads));
todo!("Actually listen")
}
async fn root() {
todo!()
}
async fn get_post_by_id(Path(post_id): Path<String>) -> impl IntoResponse {
todo!()
}
async fn get_posts_threads(Path(post_id): Path<String>, Path(discussion_thread_ids): Path<String>) -> impl IntoResponse {
let discussion_thread_ids = discussion_thread_ids.split("/");
todo!()
}
```]
As it can be seen, a parameter can be extracted from the URL path by using `/:some_name` and can be placed as any infix of the URL: at the beginning, in the middle, or the end.
However, as the path is split by slash characters (`/`), the extracted part must not contain slashes and hence can only be (up to) one part of the path.
Longer parts of the request can be obtained using `+some_name` and can only be placed as a postfix of the URL, but not other types of infixes.
This can be used to obtain the remaining path of the request.
=== Problem Description
+ The API to implement e.g. has the concept of mount points (similar to how filesystems can be mounted on UNIX-like systems).
Mount points can contain multiple slashes.
For example `/v1/some/mount/point/data/some/path/secret` may consist of a mount point `some/mount/point` and further, following routes of the mapped secret engine. //TODO
In this example, `/data` is related to the Key-Value engine and `/some/path/secret` specifies a path within the secret engine instance mounted at the mount point.
This implies a significant problem:
How to determinate what part of the URL displays a mount point, where it is not certan, what the postfix of the URL will be (this problem follows).
+ Another problem is that based on the mount point, the request must be processed by the applicable secret engine.
Based on context, stored on the DBMS, the request must be passed to the secret engine along with the determinated mount point.
=== Solution
Secret Engines have their dedicated router.
The main router has an instance of these routers along with the database pool wrapped within a struct in its state.
The router instances have a reference to the database pool, which is internally wrapped by an Atomic Reference Counter (`Arc`).
Upon a requst, the remaining path is obtained (via `/+mount_path`).
Then, the path is looked up at the database, also requsting the engine type.
If not found, the last last slash character and the following string is removed and looked up again. This is repeated, until the path either is found or has a length of zero, rejecting the request as "404 Not Found".
If found, the router is called with the request and mount path is given to the router as an "Extension".
The `call` #link("https://docs.rs/tower/0.4.13/tower/trait.Service.html#tymethod.call")[(link)]
function which Axum routers inherit from the Tower crate, allows to hand the request over to engine's router.

View file

@ -0,0 +1,35 @@
== System Design
=== Clients
The rvault server is compliant with any client acting in compliance with the hashicorp vault api specification.
To achieve this, tests are written using the official Hashicorp vault go client.
=== Webserver
The Webserver forwards client requests to different routers according to the request paths:
i.e. "/v1/auth" for the authorization router.
Those routers map requests to their corresponding handlers.
Problem:
- TODO describe middleware
=== Engines
Engines are used to manage data. Depending on the engine this can include storage, generation and encryption. Each engine is independent from other engines.
Different engines are addressed via a path, which is passed as a prefix to the secret path. The path for the key-value storage would be .../kv-v2/foo.
In the code, engines are represented by folders which are divided into a logic section, struct sections and a test section.
=== Storage
Data is currently stored in an SQLite databse which is accessed with SQLX without ORM. Later stages of the project will allow for different storage systems to be used.
=== Overview
#figure(
image("../../assets/Design.svg", width: 80%),
caption: [
The acting components of rvault.
],
)

22
documentation.typ Normal file
View file

@ -0,0 +1,22 @@
#import "@preview/arkheion:0.1.0": arkheion, arkheion-appendices
#show: arkheion.with(
title: [Secret Management with rvault],
authors: (
(name: "someone", email: "", affiliation: "HPE"),
(name: "someone", email: "", affiliation: "HPE"),
(name: "(someone)", email: "", affiliation: "HPE"),
),
// Insert your abstract after the colon, wrapped in brackets.
// Example: `abstract: [This is my abstract...]`
abstract: [The goal of this project is to implement the HashiCorp Vault in Rust. During this project we have developed an architecture and started implementation of some cases for key-value-storage. Further development is needed to achieve feature parity with the original version written in Go.],
// keywords: ("First keyword", "Second keyword", "etc."),
date: "2024-06-02",
)
#show link:underline
//#include "./chapters/01-Project-Requirements.typ"
#include "./chapters/02-Project-Goal.typ"
//#include "./chapters/03-Planning-and-Timeline.typ"
#include "./chapters/04-Architecture.typ"
#include "./chapters/05-API-Compliance-and-Testing.typ"

21
rvault_description.txt Normal file
View file

@ -0,0 +1,21 @@
Secret Management with rvault
Groupmembers:
[redacted]
Description:
The goal is to rewrite a part of the Hashicorp vault in Rust.
It should be compatible with the current version of the vault an its clients.
Therefore a vault client written in go is used to perform tests.
During this project only a small part will be implemented.
It will be further developed as an open-source project.
Implemented features include:
- Design of the architecture
- Implementation of dynamic routing to allow for exchangeable secret engines
- Basic kv-store
The main development branch is the engine-kv branch.
To execute the tests navigate to rvault/go_client/tests and execute "go test -v ./..."
To clear the test cache use "go clean -testcache"