Commit Graph

48 Commits

Author SHA1 Message Date
Stanley Dimant
7639066249 Split AuthService/Server
some refactoring
fix some stuff
add http context accessor
configure metrics as well
commit 713d054ccb965f7adb8eafa6e3fb52853a1e6dd2 (partial, Docker only)
2024-10-08 08:59:50 +00:00
Loporrit
c400d9746f Pre-fetch cache files upon pre-request 2024-09-29 14:13:59 +00:00
Loporrit
49695b4403 Use sendfile for client file get as well 2024-09-13 16:17:13 +00:00
Loporrit
b51cd61e4b Propagate touch requests via intermediate servers 2024-09-11 06:47:41 +00:00
Loporrit
b4ac5f6917 Remote touch service for fileserver shards 2024-09-08 11:22:38 +00:00
Loporrit
584f5356d7 Allow shards to use cold storage, revise cleanup service 2024-09-06 08:54:38 +00:00
Loporrit
649f3018b1 Rename MainFileCleanupService.cs for better git tracking in the next commit 2024-09-01 15:49:59 +00:00
Stanley Dimant
fb22a267ac add central cold storage options for file servers
add access times I guess

access, not write

overwrite times after copying
2024-08-27 15:07:19 +00:00
rootdarkarchon
012ae04dce add some more metrics for files downloading from caches 2024-08-27 15:07:18 +00:00
rootdarkarchon
d54d793575 refactor file server 2024-08-27 14:31:41 +00:00
rootdarkarchon
d40730bca3 some file server fixes I guess
break, not return

fix queue processing

fix queue processing, again

fix queue processing, the third

do not use async for queue processing

something
2024-08-27 14:22:10 +00:00
Stanley Dimant
72df676c9d update server to net 8
up conn semaphore

add http logging to files

update prometheus?

downgrade again to net7?

temp downgrade to net7 again?

net8 for static file server attempt #2

loporrit: MareAPI .NET8 update
2024-06-11 07:31:20 +00:00
rootdarkarchon
255798c0c6 add gauges for queue free/active/inactive 2023-08-02 10:14:27 +02:00
rootdarkarchon
4024754b47 fix counters 2023-08-01 22:15:15 +02:00
rootdarkarchon
32e9cf7795 add request size counters 2023-08-01 22:09:45 +02:00
rootdarkarchon
48323d68b0 [Draft] Update 0.8 (#25)
* get rid of file handling through grpc and signalr

* fix upload on controller

* adapt usersetpairpermissions

* send user perms

* server-side fixes

* rework file upload

* adjust log level to debug in docker standalone json

* update dependencies

---------

Co-authored-by: rootdarkarchon <root.darkarchon@outlook.com>
2023-03-14 19:46:59 +01:00
rootdarkarchon
aa03d400d4 - reload configs on change
- remove grpc config service replace with http
- add messaging to channel
2023-02-25 12:20:59 +01:00
rootdarkarchon
6f9bacab5b Messagepack (#24)
* add messagepack to signalr

* add resolvers to messagepack

---------

Co-authored-by: rootdarkarchon <root.darkarchon@outlook.com>
2023-02-10 12:30:03 +01:00
rootdarkarchon
4bd71a5889 Download rework (#22)
* rework server to send download ready back via signalr

* adjust queue handling for removal

* adjust api to main

Co-authored-by: rootdarkarchon <root.darkarchon@outlook.com>
2023-01-15 16:00:38 +01:00
rootdarkarchon
9605be8d4a minor stylefixes 2023-01-14 12:54:11 +01:00
rootdarkarchon
42b15cb6b7 Add Server-Side Download Queue (#21)
* test add queueing to file service

* further adjustments to download queueing

* add check for whether the request is still in the queue to CheckQueue

* forcefully release slot if download didn't finish in 15s

* actually cancel the delay task

* add metrics and refactor some of the request queue service

* refactor pathing

* reuse httpclient

* add queue request dto to requestfile, enqueue users immediately if a slot is available

* change startup to include all controllers

* update server pathing

* update pathing, again

* several adjustments to auth, banning, jwt server tokens, renaming, authorization

* update api I guess

* adjust automated banning of charaident and reg

* generate jwt on servers for internal authentication

* remove mvcextensions

Co-authored-by: rootdarkarchon <root.darkarchon@outlook.com>
2023-01-11 12:22:22 +01:00
rootdarkarchon
61111d387a get kestrel server options from config 2023-01-07 23:44:35 +01:00
rootdarkarchon
74b7fcdf89 Refactoring using Claims more, add Server Side Messaging (#20)
* add some refactoring based on claims, handle chara ident inside claim, fix discord userid in log

* improve authentication responses, add server side messaging

* update server to mainline api

Co-authored-by: rootdarkarchon <root.darkarchon@outlook.com>
2023-01-04 15:49:18 +01:00
rootdarkarchon
5f0c12ecfa Switch to JWT authentication (#19)
* switch to jwt authentication

* fix delete files

* adjust saving of deletion of all files

* update api to main/jwt

Co-authored-by: rootdarkarchon <root.darkarchon@outlook.com>
2023-01-02 17:07:34 +01:00
rootdarkarchon
bdd8830c8e add healthchecks to docker 2023-01-02 02:50:33 +01:00
rootdarkarchon
de2defe692 rework MareConfigurationServiceClient 2022-12-28 04:24:53 +01:00
rootdarkarchon
9eb5967935 rework server responsibilities (#18)
* rework server responsibilities
add remote configuration

* start metrics only when compiled as not debug

* add some more logging to discord bot

* fixes of some casts

* make metrics port configurable, minor fixes

* add docker bullshit

* md formatting

* adjustments to docker stuff

* fix docker json files, fix some stuff in discord bot, add /useradd for Discord bot

* adjust docker configs and fix sharded.bat

* fixes for logs, cache file provider repeat trying to open filestream

Co-authored-by: rootdarkarchon <root.darkarchon@outlook.com>
2022-12-27 13:48:05 +01:00
rootdarkarchon
7ee7fdaf48 switch around all iconfiguration usage to ioptions 2022-12-23 01:53:21 +01:00
rootdarkarchon
61af3a3da5 add metrics to CachedFileProvider 2022-12-20 14:56:45 +01:00
rootdarkarchon
25aa01c819 rework static files server 2022-12-20 14:20:24 +01:00
rootdarkarchon
ea19231851 remove unnecessary usings from solution, fixes in metrics 2022-12-20 12:00:23 +01:00
rootdarkarchon
7d3297e1e8 add some file statistics metrics 2022-12-20 05:02:03 +01:00
rootdarkarchon
ea344468f2 add http metrics 2022-12-19 17:10:19 +01:00
rootdarkarchon
827acf0282 change StaticFilesServer to use Controller 2022-12-19 16:01:25 +01:00
rootdarkarchon
90ccfe4162 adjust metrics 2022-12-19 15:01:12 +01:00
rootdarkarchon
5c9b15545c add metrics to each shard 2022-12-19 14:12:25 +01:00
rootdarkarchon
f278c5a762 rework authentication 2022-12-18 14:53:44 +01:00
rootdarkarchon
3c4344a504 allow for secondary static files server instances to be spawned 2022-12-12 10:26:29 +01:00
rootdarkarchon
c98e2b2dd6 Switch Authentication to asynchronous streaming calls (#16)
* add base grpc service and swap auth service to streaming

* remove Authorize from hub itself

* remove unused usings

* heave files server to net 7, add exception handling in grpc auth stream

Co-authored-by: rootdarkarchon <root.darkarchon@outlook.com>
2022-10-13 16:55:23 +02:00
Stanley Dimant
bc9fc95e2f adjust metrics server ports 2022-08-26 02:40:23 +02:00
Stanley Dimant
ace31926db rework metrics 2022-08-26 02:22:19 +02:00
Stanley Dimant
04cbc2bd64 fix up startup for server/fileserver and change commands to be guild commands for the discord bot + clean up unauthorized vanity uid users 2022-08-25 13:07:05 +02:00
Stanley Dimant
2f1ef04f6a add retry policy for grpc clients 2022-08-25 11:27:57 +02:00
Stanley Dimant
5b136685db fixes for standard configs 2022-08-24 21:43:17 +02:00
Stanley Dimant
74209b3952 fixes etc 2022-08-23 13:02:28 +02:00
Stanley Dimant
2510cce711 adjustments to config and cleanup nuget, remove implicit usings 2022-08-23 11:21:21 +02:00
Stanley Dimant
a63174009f decouple fileservice to be able to run standalone 2022-08-23 02:28:04 +02:00
rootdarkarchon
f9e4fd4f2d minor refactoring 2022-08-22 14:24:47 +02:00