rework server responsibilities (#18)

* rework server responsibilities
add remote configuration

* start metrics only when compiled as not debug

* add some more logging to discord bot

* fixes of some casts

* make metrics port configurable, minor fixes

* add docker bullshit

* md formatting

* adjustments to docker stuff

* fix docker json files, fix some stuff in discord bot, add /useradd for Discord bot

* adjust docker configs and fix sharded.bat

* fixes for logs, cache file provider repeat trying to open filestream

Co-authored-by: rootdarkarchon <root.darkarchon@outlook.com>

Docker/Readme.md

@@ -0,0 +1,38 @@
+# Mare Synchronos Docker Setup
+This is primarily aimed at developers who want to spin up their own local server for development purposes without having to spin up a VM.
+Obligatory requires Docker to be installed on the machine.
+
+There are two directories: `build` and `run`
+
+## 1. build images
+There is two ways to build the necessary docker images which are differentiated by the folders `-local` and `-git`  
+- -local will run the image build against the current locally present sources  
+- -git will run the image build against the latest git main commit  
+It is possible to build all required images at once by running `docker-build.bat/sh` (Server, Servies, StaticFilesServer) or all 3 separately with `docker-build-<whatever>.bat/sh`
+
+## 2. Configure ports + token
+You should set up 2 environment variables that hold server specific configuration and open up ports.  
+The default ports used through the provided configuration are `6000` for the main server and `6200` for the files downloads.  
+Both ports should be open to your computer through your router if you wish to test this with clients.
+
+Furthermore there are two environment variables `DEV_MARE_CDNURL` and `DEV_MARE_DISCORDTOKEN` which you are required to set.  
+`DEV_MARE_CDNURL` should point to `http://<yourip or dyndns>:6200/cache/` and `DEV_MARE_DISCORDTOKEN` is an oauth token from a bot you need to create through the Discord bot portal.  
+It is enough to set them as User variables. The compose files refer to those environment variables to overwrite configuration settings for the Server and Services to set those respective values.  
+It is also possible to set those values in the configuration.json files themselves.  
+Without a valid Discord bot you will not be able to register accounts without fumbling around in the PostgreSQL database.
+
+## 3. Run Mare Server
+The run folder contains two major Mare configurations which is `standalone` and `sharded`.  
+Both configurations default to port `6000` for the main server connection and `6200` for the files downloads. No HTTPS.  
+All `appsettings.json` configurations provided are extensive at the point of writing, note the differences between the shard configurations and the main servers respectively.  
+They can be used as examples if you want to spin up your own servers otherwise.
+
+The scripts to start the respective services are divided by name, the `daemon-start/stop` files use `compose up -d` to run it in the background and to be able to stop the containers as well.  
+The respective docker-compose files lie in the `compose` folder. I would not recommend editing them unless you know what you are doing.  
+All data (postgresql and files uploads) will be thrown into the `data` folder after startup.  
+All logs from the mare services will be thrown into `logs`, divided by shard, where applicable.
+
+The `standalone` configuration features PostgeSQL, Mare Server, Mare StaticFilesServer and Mare Services.  
+The `sharded` configuration features PostgreSQL, Redis, HAProxy, Mare Server Main, 2 Mare Server Shards, Mare Services, Mare StaticFilesServer Main and 2 Mare StaticFilesServer Shards.  
+Haproxy is set up that it takes the same ports as the `standalone` configuration and distributes the connections between the shards.  
+In theory it should be possible to switch between the `standalone` and `sharded` configuration by shutting down one composition container and starting up the other. They share the same Database.

Docker/build/Dockerfile-MareSynchronosServer

@@ -0,0 +1,31 @@
+FROM mcr.microsoft.com/dotnet/sdk:7.0 as BUILD
+
+COPY MareAPI /server/MareAPI
+COPY MareSynchronosServer/MareSynchronosShared /server/MareSynchronosServer/MareSynchronosShared
+COPY MareSynchronosServer/MareSynchronosServer /server/MareSynchronosServer/MareSynchronosServer
+
+WORKDIR /server/MareSynchronosServer/MareSynchronosServer/
+
+RUN dotnet publish \
+        --configuration=Release \
+        --os=linux \
+        --output=/build \
+        MareSynchronosServer.csproj
+
+FROM mcr.microsoft.com/dotnet/aspnet:7.0
+
+RUN adduser \
+        --disabled-password \
+        --group \
+        --no-create-home \
+        --quiet \
+        --system \
+        mare
+
+COPY --from=BUILD /build /opt/MareSynchronosServer
+RUN chown -R mare:mare /opt/MareSynchronosServer
+
+USER mare:mare
+WORKDIR /opt/MareSynchronosServer
+
+CMD ["./MareSynchronosServer"]

Docker/build/Dockerfile-MareSynchronosServer-git

@@ -0,0 +1,29 @@
+FROM mcr.microsoft.com/dotnet/sdk:7.0 as BUILD
+
+RUN git clone --recurse-submodules https://github.com/Penumbra-Sync/server
+
+WORKDIR /server/MareSynchronosServer/MareSynchronosServer/
+
+RUN dotnet publish \
+        --configuration=Release \
+        --os=linux \
+        --output=/MareSynchronosServer \
+        MareSynchronosServer.csproj
+
+FROM mcr.microsoft.com/dotnet/aspnet:7.0
+
+RUN adduser \
+        --disabled-password \
+        --group \
+        --no-create-home \
+        --quiet \
+        --system \
+        mare
+
+COPY --from=BUILD /MareSynchronosServer /opt/MareSynchronosServer
+RUN chown -R mare:mare /opt/MareSynchronosServer
+
+USER mare:mare
+WORKDIR /opt/MareSynchronosServer
+
+CMD ["./MareSynchronosServer"]

Docker/build/Dockerfile-MareSynchronosServices

@@ -0,0 +1,31 @@
+FROM mcr.microsoft.com/dotnet/sdk:7.0 as BUILD
+
+COPY MareAPI /server/MareAPI
+COPY MareSynchronosServer/MareSynchronosShared /server/MareSynchronosServer/MareSynchronosShared
+COPY MareSynchronosServer/MareSynchronosServices /server/MareSynchronosServer/MareSynchronosServices
+
+WORKDIR /server/MareSynchronosServer/MareSynchronosServices/
+
+RUN dotnet publish \
+        --configuration=Release \
+        --os=linux \
+        --output=/build \
+        MareSynchronosServices.csproj
+
+FROM mcr.microsoft.com/dotnet/aspnet:7.0
+
+RUN adduser \
+        --disabled-password \
+        --group \
+        --no-create-home \
+        --quiet \
+        --system \
+        mare
+
+COPY --from=BUILD /build /opt/MareSynchronosServices
+RUN chown -R mare:mare /opt/MareSynchronosServices
+
+USER mare:mare
+WORKDIR /opt/MareSynchronosServices
+
+CMD ["./MareSynchronosServices"]

Docker/build/Dockerfile-MareSynchronosServices-git

@@ -0,0 +1,29 @@
+FROM mcr.microsoft.com/dotnet/sdk:7.0 as BUILD
+
+RUN git clone --recurse-submodules https://github.com/Penumbra-Sync/server
+
+WORKDIR /server/MareSynchronosServer/MareSynchronosServices/
+
+RUN dotnet publish \
+        --configuration=Release \
+        --os=linux \
+        --output=/MareSynchronosServices \
+        MareSynchronosServices.csproj
+
+FROM mcr.microsoft.com/dotnet/aspnet:7.0
+
+RUN adduser \
+        --disabled-password \
+        --group \
+        --no-create-home \
+        --quiet \
+        --system \
+        mare
+
+COPY --from=BUILD /MareSynchronosServices /opt/MareSynchronosServices
+RUN chown -R mare:mare /opt/MareSynchronosServices
+
+USER mare:mare
+WORKDIR /opt/MareSynchronosServices
+
+CMD ["./MareSynchronosServices"]

Docker/build/Dockerfile-MareSynchronosStaticFilesServer

@@ -0,0 +1,31 @@
+FROM mcr.microsoft.com/dotnet/sdk:7.0 as BUILD
+
+COPY MareAPI /server/MareAPI
+COPY MareSynchronosServer/MareSynchronosShared /server/MareSynchronosServer/MareSynchronosShared
+COPY MareSynchronosServer/MareSynchronosStaticFilesServer /server/MareSynchronosServer/MareSynchronosStaticFilesServer
+
+WORKDIR /server/MareSynchronosServer/MareSynchronosStaticFilesServer/
+
+RUN dotnet publish \
+        --configuration=Release \
+        --os=linux \
+        --output=/build \
+        MareSynchronosStaticFilesServer.csproj
+
+FROM mcr.microsoft.com/dotnet/aspnet:7.0
+
+RUN adduser \
+        --disabled-password \
+        --group \
+        --no-create-home \
+        --quiet \
+        --system \
+        mare
+
+COPY --from=BUILD /build /opt/MareSynchronosStaticFilesServer
+RUN chown -R mare:mare /opt/MareSynchronosStaticFilesServer
+
+USER mare:mare
+WORKDIR /opt/MareSynchronosStaticFilesServer
+
+CMD ["./MareSynchronosStaticFilesServer"]

Docker/build/Dockerfile-MareSynchronosStaticFilesServer-git

@@ -0,0 +1,29 @@
+FROM mcr.microsoft.com/dotnet/sdk:7.0 as BUILD
+
+RUN git clone --recurse-submodules https://github.com/Penumbra-Sync/server
+
+WORKDIR /server/MareSynchronosServer/MareSynchronosStaticFilesServer/
+
+RUN dotnet publish \
+        --configuration=Release \
+        --os=linux \
+        --output=/MareSynchronosStaticFilesServer \
+        MareSynchronosStaticFilesServer.csproj
+
+FROM mcr.microsoft.com/dotnet/aspnet:7.0
+
+RUN adduser \
+        --disabled-password \
+        --group \
+        --no-create-home \
+        --quiet \
+        --system \
+        mare
+
+COPY --from=BUILD /MareSynchronosStaticFilesServer /opt/MareSynchronosStaticFilesServer
+RUN chown -R mare:mare /opt/MareSynchronosStaticFilesServer
+
+USER mare:mare
+WORKDIR /opt/MareSynchronosStaticFilesServer
+
+CMD ["./MareSynchronosStaticFilesServer"]

Docker/build/linux-git/docker-build-server.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+docker build -t darkarchon/mare-synchronos-server:latest . -f ../Dockerfile-MareSynchronosServer-git --no-cache --pull --force-rm

Docker/build/linux-git/docker-build-services.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+docker build -t darkarchon/mare-synchronos-services:latest . -f ../Dockerfile-MareSynchronosServices-git --no-cache --pull --force-rm

Docker/build/linux-git/docker-build-staticfilesserver.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+docker build -t darkarchon/mare-synchronos-staticfilesserver:latest . -f ../Dockerfile-MareSynchronosStaticFilesServer-git --no-cache --pull --force-rm

Docker/build/linux-git/docker-build.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+./docker-build-server.sh
+./docker-build-services.sh
+./docker-build-staticfilesserver.sh

Docker/build/linux-local/docker-build-server.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+cd ../../../
+docker build -t darkarchon/mare-synchronos-server:latest . -f ../Dockerfile-MareSynchronosServer --no-cache --pull --force-rm
+cd Docker/build/linux-local

Docker/build/linux-local/docker-build-services.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+cd ../../../
+docker build -t darkarchon/mare-synchronos-services:latest . -f ../Dockerfile-MareSynchronosServices --no-cache --pull --force-rm
+cd Docker/build/linux-local

Docker/build/linux-local/docker-build-staticfilesserver.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+cd ../../../
+docker build -t darkarchon/mare-synchronos-staticfilesserver:latest . -f ../Dockerfile-MareSynchronosStaticFilesServer --no-cache --pull --force-rm
+cd Docker/build/linux-local

Docker/build/linux-local/docker-build.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+./docker-build-server.sh
+./docker-build-services.sh
+./docker-build-staticfilesserver.sh

Docker/build/windows-git/docker-build-server.bat

@@ -0,0 +1,2 @@
+@echo off
+docker build -t darkarchon/mare-synchronos-server:latest . -f ..\Dockerfile-MareSynchronosServer-git --no-cache --pull --force-rm

Docker/build/windows-git/docker-build-services.bat

@@ -0,0 +1,3 @@
+@echo off
+
+docker build -t darkarchon/mare-synchronos-services:latest . -f ..\Dockerfile-MareSynchronosServices-git --no-cache --pull --force-rm

Docker/build/windows-git/docker-build-staticfilesserver.bat

@@ -0,0 +1,3 @@
+@echo off
+
+docker build -t darkarchon/mare-synchronos-staticfilesserver:latest . -f ..\Dockerfile-MareSynchronosStaticFilesServer-git --no-cache --pull --force-rm

Docker/build/windows-git/docker-build.bat

@@ -0,0 +1,5 @@
+@echo off
+
+call docker-build-server.bat
+call docker-build-services.bat
+call docker-build-staticfilesserver.bat

Docker/build/windows-local/docker-build-server.bat

@@ -0,0 +1,4 @@
+@echo off
+cd ..\..\..\
+docker build -t darkarchon/mare-synchronos-server:latest . -f Docker\build\Dockerfile-MareSynchronosServer --no-cache --pull --force-rm
+cd Docker\build\windows-local

Docker/build/windows-local/docker-build-services.bat

@@ -0,0 +1,4 @@
+@echo off
+cd ..\..\..\
+docker build -t darkarchon/mare-synchronos-services:latest . -f Docker\build\Dockerfile-MareSynchronosServices --no-cache --pull --force-rm
+cd Docker\build\windows-local

Docker/build/windows-local/docker-build-staticfilesserver.bat

@@ -0,0 +1,4 @@
+@echo off
+cd ..\..\..\
+docker build -t darkarchon/mare-synchronos-staticfilesserver:latest . -f Docker\build\Dockerfile-MareSynchronosStaticFilesServer --no-cache --pull --force-rm
+cd Docker\build\windows-local

Docker/build/windows-local/docker-build.bat

@@ -0,0 +1,5 @@
+@echo off
+
+call docker-build-server.bat
+call docker-build-services.bat
+call docker-build-staticfilesserver.bat

Docker/run/compose/mare-sharded.yml

@@ -0,0 +1,114 @@
+services:
+  postgres:
+    image: postgres:latest
+    restart: always
+    environment:
+      POSTGRES_DB: mare
+      POSTGRES_USER: mare
+      POSTGRES_PASSWORD: secretdevpassword
+    volumes:
+      - ../data/postgresql/:/var/lib/postgresql/data
+      - postgres_socket:/var/run/postgresql:rw
+
+  haproxy:
+    image: haproxy:latest
+    restart: always
+    ports: 
+      - 6000:6000/tcp
+      - 6200:6200/tcp
+    volumes:
+      - ../config/sharded/haproxy-shards.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
+
+  redis:
+    image: redis:latest
+    command: [sh, -c, "rm -f /data/dump.rdb && redis-server --save \"\" --appendonly no --requirepass secretredispassword"]
+    volumes:
+      - cache:/data
+
+  mare-server:
+    image: darkarchon/mare-synchronos-server:latest
+    restart: on-failure
+    environment:
+      MareSynchronos__CdnFullUrl: "${DEV_MARE_CDNURL}"
+    volumes:
+      - ../config/sharded/server-shard-main.json:/opt/MareSynchronosServer/appsettings.json
+      - ../log/server-shard-main/:/opt/MareSynchronosServer/logs/:rw
+      - postgres_socket:/var/run/postgresql/:rw
+    depends_on:
+      - "postgres"
+
+  mare-shard-1:
+    image: darkarchon/mare-synchronos-server:latest
+    restart: on-failure
+    volumes:
+      - ../config/sharded/server-shard-1.json:/opt/MareSynchronosServer/appsettings.json
+      - ../log/server-shard-1/:/opt/MareSynchronosServer/logs/:rw
+      - postgres_socket:/var/run/postgresql/:rw
+    depends_on:
+      - "postgres"
+      - "mare-server"
+
+  mare-shard-2:
+    image: darkarchon/mare-synchronos-server:latest
+    restart: on-failure
+    volumes:
+      - ../config/sharded/server-shard-2.json:/opt/MareSynchronosServer/appsettings.json
+      - ../log/server-shard-2/:/opt/MareSynchronosServer/logs/:rw
+      - postgres_socket:/var/run/postgresql/:rw
+    depends_on:
+      - "postgres"
+      - "mare-server"
+
+  mare-services:
+    image: darkarchon/mare-synchronos-services:latest
+    restart: on-failure
+    environment:
+      MareSynchronos__DiscordBotToken: "${DEV_MARE_DISCORDTOKEN}"
+    volumes:
+      - ../config/standalone/services-standalone.json:/opt/MareSynchronosServices/appsettings.json
+      - ../log/services-standalone/:/opt/MareSynchronosServices/logs/:rw
+      - postgres_socket:/var/run/postgresql/:rw
+    depends_on:
+      - "postgres"
+      - "mare-server"
+
+  mare-files:
+    image: darkarchon/mare-synchronos-staticfilesserver:latest
+    restart: on-failure
+    volumes:
+      - ../config/sharded/files-shard-main.json:/opt/MareSynchronosStaticFilesServer/appsettings.json
+      - ../log/files-standalone/:/opt/MareSynchronosStaticFilesServer/logs/:rw
+      - postgres_socket:/var/run/postgresql/:rw
+      - ../data/files-shard-main/:/marecache/:rw
+    depends_on:
+      - "postgres"
+      - "mare-server"
+
+  mare-files-shard-1:
+    image: darkarchon/mare-synchronos-staticfilesserver:latest
+    restart: on-failure
+    volumes:
+      - ../config/sharded/files-shard-1.json:/opt/MareSynchronosStaticFilesServer/appsettings.json
+      - ../log/files-shard-1/:/opt/MareSynchronosStaticFilesServer/logs/:rw
+      - postgres_socket:/var/run/postgresql/:rw
+      - ../data/files-shard-1/:/marecache/:rw
+    depends_on:
+      - "postgres"
+      - "mare-files"
+
+  mare-files-shard-2:
+    image: darkarchon/mare-synchronos-staticfilesserver:latest
+    restart: on-failure
+    volumes:
+      - ../config/sharded/files-shard-2.json:/opt/MareSynchronosStaticFilesServer/appsettings.json
+      - ../log/files-shard-2/:/opt/MareSynchronosStaticFilesServer/logs/:rw
+      - postgres_socket:/var/run/postgresql/:rw
+      - ../data/files-shard-2/:/marecache/:rw
+    depends_on:
+      - "postgres"
+      - "mare-files"
+
+volumes:
+  cache:
+    driver: local
+  postgres_socket:

Docker/run/compose/mare-standalone.yml

@@ -0,0 +1,55 @@
+services:
+  postgres:
+    image: postgres:latest
+    restart: always
+    environment:
+      POSTGRES_DB: mare
+      POSTGRES_USER: mare
+      POSTGRES_PASSWORD: secretdevpassword
+    volumes:
+      - ../data/postgresql/:/var/lib/postgresql/data
+      - postgres_socket:/var/run/postgresql:rw
+
+  mare-server:
+    image: darkarchon/mare-synchronos-server:latest
+    restart: on-failure
+    ports:
+      - 6000:6000/tcp
+    environment:
+      MareSynchronos__CdnFullUrl: "${DEV_MARE_CDNURL}"
+    volumes:
+      - ../config/standalone/server-standalone.json:/opt/MareSynchronosServer/appsettings.json
+      - ../log/server-standalone/:/opt/MareSynchronosServer/logs/:rw
+      - postgres_socket:/var/run/postgresql/:rw
+    depends_on:
+      - "postgres"
+
+  mare-services:
+    image: darkarchon/mare-synchronos-services:latest
+    restart: on-failure
+    environment:
+      MareSynchronos__DiscordBotToken: "${DEV_MARE_DISCORDTOKEN}"
+    volumes:
+      - ../config/standalone/services-standalone.json:/opt/MareSynchronosServices/appsettings.json
+      - ../log/services-standalone/:/opt/MareSynchronosServices/logs/:rw
+      - postgres_socket:/var/run/postgresql/:rw
+    depends_on:
+      - "postgres"
+      - "mare-server"
+
+  mare-files:
+    image: darkarchon/mare-synchronos-staticfilesserver:latest
+    ports:
+      - 6200:6200/tcp
+    restart: on-failure
+    volumes:
+      - ../config/standalone/files-standalone.json:/opt/MareSynchronosStaticFilesServer/appsettings.json
+      - ../log/files-standalone/:/opt/MareSynchronosStaticFilesServer/logs/:rw
+      - postgres_socket:/var/run/postgresql/:rw
+      - ../data/files-standalone/:/marecache/:rw
+    depends_on:
+      - "postgres"
+      - "mare-server"
+
+volumes:
+  postgres_socket:

Docker/run/config/sharded/files-shard-1.json

@@ -0,0 +1,49 @@
+{
+    "ConnectionStrings": {
+        "DefaultConnection": "Host=/var/run/postgresql;Port=5432;Database=mare;Username=mare;Keepalive=15;Minimum Pool Size=10;Maximum Pool Size=50;No Reset On Close=true;Max Auto Prepare=50;Enlist=false"
+    },
+    "Logging": {
+        "LogLevel": {
+            "Default": "Warning",
+            "Microsoft": "Warning",
+            "Microsoft.Hosting.Lifetime": "Information",
+            "MareSynchronosStaticFilesServer": "Information",
+            "MareSynchronosShared": "Information",
+            "System.IO": "Information"
+        },
+        "File": {
+            "BasePath": "logs",
+            "FileAccessMode": "KeepOpenAndAutoFlush",
+            "FileEncodingName": "utf-8",
+            "DateFormat": "yyyMMdd",
+            "MaxFileSize": 104857600,
+            "Files": [
+                {
+                    "Path": "<date:yyyy>/<date:MM>/<date:dd>/mare-<date:HH>-<counter:0000>.log"
+                }
+            ]
+        }
+    },
+    "MareSynchronos": {
+        "DbContextPoolSize": 512,
+        "ShardName": "Files Shard 1",
+        "MetricsPort": 6250,
+        "FileServerGrpcAddress": "http://mare-files:6205",
+        "ForcedDeletionOfFilesAfterHours": 2,
+        "CacheSizeHardLimitInGiB": 5,
+        "UnusedFileRetentionPeriodInDays": 14,
+        "CacheDirectory": "/marecache/",
+        "RemoteCacheSourceUri": "http://mare-files:6200/cache/",
+        "MainServerGrpcAddress": "http://mare-server:6005"
+    },
+    "AllowedHosts": "*",
+    "Kestrel": {
+        "Endpoints": {
+            "Http": {
+                "Url": "http://+:6200"
+            }
+        }
+    },
+    "IpRateLimiting": {},
+    "IPRateLimitPolicies": {}
+}

Docker/run/config/sharded/files-shard-2.json

@@ -0,0 +1,49 @@
+{
+    "ConnectionStrings": {
+        "DefaultConnection": "Host=/var/run/postgresql;Port=5432;Database=mare;Username=mare;Keepalive=15;Minimum Pool Size=10;Maximum Pool Size=50;No Reset On Close=true;Max Auto Prepare=50;Enlist=false"
+    },
+    "Logging": {
+        "LogLevel": {
+            "Default": "Warning",
+            "Microsoft": "Warning",
+            "Microsoft.Hosting.Lifetime": "Information",
+            "MareSynchronosStaticFilesServer": "Information",
+            "MareSynchronosShared": "Information",
+            "System.IO": "Information"
+        },
+        "File": {
+            "BasePath": "logs",
+            "FileAccessMode": "KeepOpenAndAutoFlush",
+            "FileEncodingName": "utf-8",
+            "DateFormat": "yyyMMdd",
+            "MaxFileSize": 104857600,
+            "Files": [
+                {
+                    "Path": "<date:yyyy>/<date:MM>/<date:dd>/mare-<date:HH>-<counter:0000>.log"
+                }
+            ]
+        }
+    },
+    "MareSynchronos": {
+        "DbContextPoolSize": 512,
+        "ShardName": "Files Shard 2",
+        "MetricsPort": 6250,
+        "FileServerGrpcAddress": "http://mare-files:6205",
+        "ForcedDeletionOfFilesAfterHours": 2,
+        "CacheSizeHardLimitInGiB": 5,
+        "UnusedFileRetentionPeriodInDays": 14,
+        "CacheDirectory": "/marecache/",
+        "RemoteCacheSourceUri": "http://mare-files:6200/cache/",
+        "MainServerGrpcAddress": "http://mare-server:6005"
+    },
+    "AllowedHosts": "*",
+    "Kestrel": {
+        "Endpoints": {
+            "Http": {
+                "Url": "http://+:6200"
+            }
+        }
+    },
+    "IpRateLimiting": {},
+    "IPRateLimitPolicies": {}
+}

Docker/run/config/sharded/files-shard-main.json

@@ -0,0 +1,53 @@
+{
+    "ConnectionStrings": {
+        "DefaultConnection": "Host=/var/run/postgresql;Port=5432;Database=mare;Username=mare;Keepalive=15;Minimum Pool Size=10;Maximum Pool Size=50;No Reset On Close=true;Max Auto Prepare=50;Enlist=false"
+    },
+    "Logging": {
+        "LogLevel": {
+            "Default": "Warning",
+            "Microsoft": "Warning",
+            "Microsoft.Hosting.Lifetime": "Information",
+            "MareSynchronosStaticFilesServer": "Information",
+            "MareSynchronosShared": "Information",
+            "System.IO": "Information"
+        },
+        "File": {
+            "BasePath": "logs",
+            "FileAccessMode": "KeepOpenAndAutoFlush",
+            "FileEncodingName": "utf-8",
+            "DateFormat": "yyyMMdd",
+            "MaxFileSize": 104857600,
+            "Files": [
+                {
+                    "Path": "<date:yyyy>/<date:MM>/<date:dd>/mare-<date:HH>-<counter:0000>.log"
+                }
+            ]
+        }
+    },
+    "MareSynchronos": {
+        "DbContextPoolSize": 512,
+        "ShardName": "Files",
+        "MetricsPort": 6250,
+        "FileServerGrpcAddress": "",
+        "ForcedDeletionOfFilesAfterHours": -1,
+        "CacheSizeHardLimitInGiB": -1,
+        "UnusedFileRetentionPeriodInDays": 14,
+        "CacheDirectory": "/marecache/",
+        "RemoteCacheSourceUri": "",
+        "MainServerGrpcAddress": "http://mare-server:6005"
+    },
+    "AllowedHosts": "*",
+    "Kestrel": {
+        "Endpoints": {
+            "Http": {
+                "Url": "http://+:6200"
+            },
+            "Grpc": {
+                "Protocols": "Http2",
+                "Url": "http://+:6205"
+            }
+        }
+    },
+    "IpRateLimiting": {},
+    "IPRateLimitPolicies": {}
+}

Docker/run/config/sharded/haproxy-shards.cfg

@@ -0,0 +1,39 @@
+global
+        log /dev/log    local0
+        log /dev/log    local1 notice
+        daemon
+
+        ca-base /etc/ssl/certs
+        crt-base /etc/ssl/private
+
+        ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
+        ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
+        ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
+
+defaults
+        log     global
+        mode    http
+        option  httplog
+        option  dontlognull
+        timeout connect 5000
+        timeout client  50000
+        timeout server  50000
+
+frontend mare
+        bind :6000
+        default_backend mare-servers
+
+frontend mare-files
+        bind :6200
+        default_backend mare-files
+
+backend mare-servers
+        balance leastconn
+        cookie SERVER insert indirect nocache
+        server mare1 mare-shard-1:6000 cookie mare1
+        server mare2 mare-shard-2:6000 cookie mare2
+
+backend mare-files
+        balance roundrobin
+        server files1 mare-files-shard-1:6200
+        server files2 mare-files-shard-2:6200

Docker/run/config/sharded/server-shard-1.json

@@ -0,0 +1,44 @@
+{
+    "ConnectionStrings": {
+        "DefaultConnection": "Host=/var/run/postgresql;Port=5432;Database=mare;Username=mare;Keepalive=15;Minimum Pool Size=10;Maximum Pool Size=50;No Reset On Close=true;Max Auto Prepare=50;Enlist=false"
+    },
+    "Logging": {
+        "LogLevel": {
+            "Default": "Warning",
+            "Microsoft": "Warning",
+            "Microsoft.Hosting.Lifetime": "Information",
+            "MareSynchronosServer": "Information",
+            "MareSynchronosShared": "Information",
+            "System.IO": "Information"
+        },
+        "File": {
+            "BasePath": "logs",
+            "FileAccessMode": "KeepOpenAndAutoFlush",
+            "FileEncodingName": "utf-8",
+            "DateFormat": "yyyMMdd",
+            "MaxFileSize": 104857600,
+            "Files": [
+                {
+                    "Path": "<date:yyyy>/<date:MM>/<date:dd>/mare-<date:HH>-<counter:0000>.log"
+                }
+            ]
+        }
+    },
+    "MareSynchronos": {
+        "DbContextPoolSize": 512,
+        "ShardName": "Shard 1",
+        "MetricsPort": 6050,
+        "MainServerGrpcAddress": "http://mare-server:6005",
+        "RedisConnectionString": "redis,password=secretredispassword"
+    },
+    "AllowedHosts": "*",
+    "Kestrel": {
+        "Endpoints": {
+            "Http": {
+                "Url": "http://+:6000"
+            }
+        }
+    },
+    "IpRateLimiting": {},
+    "IPRateLimitPolicies": {}
+}

Docker/run/config/sharded/server-shard-2.json

@@ -0,0 +1,44 @@
+{
+    "ConnectionStrings": {
+        "DefaultConnection": "Host=/var/run/postgresql;Port=5432;Database=mare;Username=mare;Keepalive=15;Minimum Pool Size=10;Maximum Pool Size=50;No Reset On Close=true;Max Auto Prepare=50;Enlist=false"
+    },
+    "Logging": {
+        "LogLevel": {
+            "Default": "Warning",
+            "Microsoft": "Warning",
+            "Microsoft.Hosting.Lifetime": "Information",
+            "MareSynchronosServer": "Information",
+            "MareSynchronosShared": "Information",
+            "System.IO": "Information"
+        },
+        "File": {
+            "BasePath": "logs",
+            "FileAccessMode": "KeepOpenAndAutoFlush",
+            "FileEncodingName": "utf-8",
+            "DateFormat": "yyyMMdd",
+            "MaxFileSize": 104857600,
+            "Files": [
+                {
+                    "Path": "<date:yyyy>/<date:MM>/<date:dd>/mare-<date:HH>-<counter:0000>.log"
+                }
+            ]
+        }
+    },
+    "MareSynchronos": {
+        "DbContextPoolSize": 512,
+        "ShardName": "Shard 2",
+        "MetricsPort": 6050,
+        "MainServerGrpcAddress": "http://mare-server:6005",
+        "RedisConnectionString": "redis,password=secretredispassword"
+    },
+    "AllowedHosts": "*",
+    "Kestrel": {
+        "Endpoints": {
+            "Http": {
+                "Url": "http://+:6000"
+            }
+        }
+    },
+    "IpRateLimiting": {},
+    "IPRateLimitPolicies": {}
+}

Docker/run/config/sharded/server-shard-main.json

@@ -0,0 +1,60 @@
+{
+    "ConnectionStrings": {
+        "DefaultConnection": "Host=/var/run/postgresql;Port=5432;Database=mare;Username=mare;Keepalive=15;Minimum Pool Size=10;Maximum Pool Size=50;No Reset On Close=true;Max Auto Prepare=50;Enlist=false"
+    },
+    "Logging": {
+        "LogLevel": {
+            "Default": "Warning",
+            "Microsoft": "Warning",
+            "Microsoft.Hosting.Lifetime": "Information",
+            "MareSynchronosServer": "Information",
+            "MareSynchronosShared": "Information",
+            "System.IO": "Information"
+        },
+        "File": {
+            "BasePath": "logs",
+            "FileAccessMode": "KeepOpenAndAutoFlush",
+            "FileEncodingName": "utf-8",
+            "DateFormat": "yyyMMdd",
+            "MaxFileSize": 104857600,
+            "Files": [
+                {
+                    "Path": "<date:yyyy>/<date:MM>/<date:dd>/mare-<date:HH>-<counter:0000>.log"
+                }
+            ]
+        }
+    },
+    "MareSynchronos": {
+        "DbContextPoolSize": 512,
+        "ShardName": "Main",
+        "MetricsPort": 6050,
+        "MainServerGrpcAddress": "",
+        "FailedAuthForTempBan": 5,
+        "TempBanDurationInMinutes": 5,
+        "WhitelistedIps": [
+            ""
+        ],
+        "RedisConnectionString": "redis,password=secretredispassword",
+        "CdnFullUrl": "http://localhost:6200/cache/",
+        "StaticFileServiceAddress": "http://mare-files:6205",
+        "MaxExistingGroupsByUser": 3,
+        "MaxJoinedGroupsByUser": 6,
+        "MaxGroupUserCount": 100,
+        "PurgeUnusedAccounts": false,
+        "PurgeUnusedAccountsPeriodInDays": 14
+    },
+    "AllowedHosts": "*",
+    "Kestrel": {
+        "Endpoints": {
+            "Http": {
+                "Url": "http://+:6000"
+            },
+            "Grpc": {
+                "Protocols": "Http2",
+                "Url": "http://+:6005"
+            }
+        }
+    },
+    "IpRateLimiting": {},
+    "IPRateLimitPolicies": {}
+}

Docker/run/config/standalone/files-standalone.json

@@ -0,0 +1,53 @@
+{
+    "ConnectionStrings": {
+        "DefaultConnection": "Host=/var/run/postgresql;Port=5432;Database=mare;Username=mare;Keepalive=15;Minimum Pool Size=10;Maximum Pool Size=50;No Reset On Close=true;Max Auto Prepare=50;Enlist=false"
+    },
+    "Logging": {
+        "LogLevel": {
+            "Default": "Warning",
+            "Microsoft": "Warning",
+            "Microsoft.Hosting.Lifetime": "Information",
+            "MareSynchronosStaticFilesServer": "Information",
+            "MareSynchronosShared": "Information",
+            "System.IO": "Information"
+        },
+        "File": {
+            "BasePath": "logs",
+            "FileAccessMode": "KeepOpenAndAutoFlush",
+            "FileEncodingName": "utf-8",
+            "DateFormat": "yyyMMdd",
+            "MaxFileSize": 104857600,
+            "Files": [
+                {
+                    "Path": "<date:yyyy>/<date:MM>/<date:dd>/mare-<date:HH>-<counter:0000>.log"
+                }
+            ]
+        }
+    },
+    "MareSynchronos": {
+        "DbContextPoolSize": 512,
+        "ShardName": "Files",
+        "MetricsPort": 6250,
+        "FileServerGrpcAddress": "",
+        "ForcedDeletionOfFilesAfterHours": -1,
+        "CacheSizeHardLimitInGiB": -1,
+        "UnusedFileRetentionPeriodInDays": 14,
+        "CacheDirectory": "/marecache/",
+        "RemoteCacheSourceUri": "",
+        "MainServerGrpcAddress": "http://mare-server:6005"
+    },
+    "AllowedHosts": "*",
+    "Kestrel": {
+        "Endpoints": {
+            "Http": {
+                "Url": "http://+:6200"
+            },
+            "Grpc": {
+                "Protocols": "Http2",
+                "Url": "http://+:6205"
+            }
+        }
+    },
+    "IpRateLimiting": {},
+    "IPRateLimitPolicies": {}
+}

Docker/run/config/standalone/server-standalone.json

@@ -0,0 +1,60 @@
+{
+    "ConnectionStrings": {
+        "DefaultConnection": "Host=/var/run/postgresql;Port=5432;Database=mare;Username=mare;Keepalive=15;Minimum Pool Size=10;Maximum Pool Size=50;No Reset On Close=true;Max Auto Prepare=50;Enlist=false"
+    },
+    "Logging": {
+        "LogLevel": {
+            "Default": "Warning",
+            "Microsoft": "Warning",
+            "Microsoft.Hosting.Lifetime": "Information",
+            "MareSynchronosServer": "Information",
+            "MareSynchronosShared": "Information",
+            "System.IO": "Information"
+        },
+        "File": {
+            "BasePath": "logs",
+            "FileAccessMode": "KeepOpenAndAutoFlush",
+            "FileEncodingName": "utf-8",
+            "DateFormat": "yyyMMdd",
+            "MaxFileSize": 104857600,
+            "Files": [
+                {
+                    "Path": "<date:yyyy>/<date:MM>/<date:dd>/mare-<date:HH>-<counter:0000>.log"
+                }
+            ]
+        }
+    },
+    "MareSynchronos": {
+        "DbContextPoolSize": 512,
+        "ShardName": "Main",
+        "MetricsPort": 6050,
+        "MainServerGrpcAddress": "",
+        "FailedAuthForTempBan": 5,
+        "TempBanDurationInMinutes": 5,
+        "WhitelistedIps": [
+            ""
+        ],
+        "RedisConnectionString": "",
+        "CdnFullUrl": "http://localhost:6200/cache/",
+        "StaticFileServiceAddress": "http://mare-files:6205",
+        "MaxExistingGroupsByUser": 3,
+        "MaxJoinedGroupsByUser": 6,
+        "MaxGroupUserCount": 100,
+        "PurgeUnusedAccounts": false,
+        "PurgeUnusedAccountsPeriodInDays": 14
+    },
+    "AllowedHosts": "*",
+    "Kestrel": {
+        "Endpoints": {
+            "Http": {
+                "Url": "http://+:6000"
+            },
+            "Grpc": {
+                "Protocols": "Http2",
+                "Url": "http://+:6005"
+            }
+        }
+    },
+    "IpRateLimiting": {},
+    "IPRateLimitPolicies": {}
+}

Docker/run/config/standalone/services-standalone.json

@@ -0,0 +1,39 @@
+{
+    "ConnectionStrings": {
+        "DefaultConnection": "Host=/var/run/postgresql;Port=5432;Database=mare;Username=mare;Keepalive=15;Minimum Pool Size=10;Maximum Pool Size=50;No Reset On Close=true;Max Auto Prepare=50;Enlist=false"
+    },
+    "Logging": {
+        "LogLevel": {
+            "Default": "Warning",
+            "Microsoft": "Warning",
+            "Microsoft.Hosting.Lifetime": "Information",
+            "MareSynchronosServices": "Information",
+            "MareSynchronosShared": "Information",
+            "System.IO": "Information"
+        },
+        "File": {
+            "BasePath": "logs",
+            "FileAccessMode": "KeepOpenAndAutoFlush",
+            "FileEncodingName": "utf-8",
+            "DateFormat": "yyyMMdd",
+            "MaxFileSize": 104857600,
+            "Files": [
+                {
+                    "Path": "<date:yyyy>/<date:MM>/<date:dd>/mare-<date:HH>-<counter:0000>.log"
+                }
+            ]
+        }
+    },
+    "MareSynchronos": {
+        "DbContextPoolSize": 512,
+        "ShardName": "Services",
+        "MetricsPort": 6150,
+        "MainServerGrpcAddress": "http://mare-server:6005",
+        "DiscordBotToken": ""
+    },
+    "AllowedHosts": "*",
+    "Kestrel": {
+    },
+    "IpRateLimiting": {},
+    "IPRateLimitPolicies": {}
+}

Docker/run/linux-sharded-daemon-start.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+docker compose -f compose/mare-sharded.yml -p sharded up -d

Docker/run/linux-sharded-daemon-stop.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+docker compose -f compose/mare-sharded.yml -p sharded stop

Docker/run/linux-sharded.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+docker compose -f compose/mare-sharded.yml -p sharded up

Docker/run/linux-standalone-daemon-start.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+docker compose -f compose/mare-standalone.yml -p standalone up -d

Docker/run/linux-standalone-daemon-stop.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+docker compose -f compose/mare-standalone.yml -p standalone stop

Docker/run/linux-standalone.sh

@@ -0,0 +1,2 @@
+#!/bin/sh
+docker compose -f compose/mare-standalone.yml -p standalone up

Docker/run/windows-sharded-daemon-start.bat

@@ -0,0 +1,2 @@
+@echo off
+docker compose -f compose\mare-sharded.yml -p sharded up -d

Docker/run/windows-sharded-daemon-stop.bat

@@ -0,0 +1,2 @@
+@echo off
+docker compose -f compose\mare-sharded.yml -p sharded stop

Docker/run/windows-sharded.bat

@@ -0,0 +1,2 @@
+@echo off
+docker compose -f compose\mare-sharded.yml -p sharded up

Docker/run/windows-standalone-daemon-start.bat

@@ -0,0 +1,2 @@
+@echo off
+docker compose -f compose\mare-standalone.yml -p standalone up -d

Docker/run/windows-standalone-daemon-stop.bat

@@ -0,0 +1,2 @@
+@echo off
+docker compose -f compose\mare-standalone.yml -p standalone stop

Docker/run/windows-standalone.bat

@@ -0,0 +1,2 @@
+@echo off
+docker compose -f compose\mare-standalone.yml -p standalone up