Commit Graph

491 Commits

Author SHA1 Message Date
Loporrit
908c34f57b Merge branch 'cold-and-shards' 2024-09-08 12:36:18 +00:00
Loporrit
b4ac5f6917 Remote touch service for fileserver shards 2024-09-08 11:22:38 +00:00
Loporrit
16f8f23545 silence, client ready 2024-09-08 11:03:05 +00:00
Loporrit
0b6af10a44 Prioritize larger files during size cleanup 2024-09-08 11:03:05 +00:00
Loporrit
1782a16ea4 Avoid race conditon between uploads and cleanup 2024-09-06 08:54:48 +00:00
Loporrit
2090c65bf1 Allow file shards to signal main server directly 2024-09-06 08:54:45 +00:00
Loporrit
524e412c28 Add DistributionFileServerForceHTTP2 2024-09-06 08:54:45 +00:00
Loporrit
c42d0b38cf Keep main server out of file business, and disable user file deletion 2024-09-06 08:54:45 +00:00
Loporrit
584f5356d7 Allow shards to use cold storage, revise cleanup service 2024-09-06 08:54:38 +00:00
Loporrit
649f3018b1 Rename MainFileCleanupService.cs for better git tracking in the next commit 2024-09-01 15:49:59 +00:00
Loporrit
fb0905ef84 check cold storage outside of download task instead 2024-08-30 08:32:55 +00:00
Loporrit
05c880d4e8 no IDbContextFactory 2024-08-30 08:24:49 +00:00
Stanley Dimant
b7e12f3e42 handle unknown continent 2024-08-27 15:07:19 +00:00
Stanley Dimant
56a257261b allow more time for download tasks in cachedfileprovider 2024-08-27 15:07:19 +00:00
Stanley Dimant
aa78432db3 more queue fixes 2024-08-27 15:07:19 +00:00
Stanley Dimant
a116545c9e requeue requests where files are still downloading 2024-08-27 15:07:19 +00:00
Stanley Dimant
9269a403b5 refactor cleanup stuff 2024-08-27 15:07:19 +00:00
Stanley Dimant
f97aa46f74 cold storage retention should be days
limit is double though jksdfjklsdfkl
2024-08-27 15:07:19 +00:00
Stanley Dimant
09662b477e clean up some code and optinmize cleanup
speed up cleanup service

fixes for fixes
2024-08-27 15:07:19 +00:00
Stanley Dimant
fb22a267ac add central cold storage options for file servers
add access times I guess

access, not write

overwrite times after copying
2024-08-27 15:07:19 +00:00
Stanley Dimant
180d7d504d add special handling for NA-W/NA-E
remove unused usings
2024-08-27 15:07:19 +00:00
Stanley Dimant
b189812dc0 help? 2024-08-27 15:07:19 +00:00
Stanley Dimant
1e353229ac some fixes maybe
kms
2024-08-27 15:07:18 +00:00
Stanley Dimant
128a5a0aa2 fix send ready message service probably 2024-08-27 15:07:18 +00:00
rootdarkarchon
c06edeebdd fix AnyFilesDownloading 2024-08-27 15:07:18 +00:00
rootdarkarchon
012ae04dce add some more metrics for files downloading from caches 2024-08-27 15:07:18 +00:00
rootdarkarchon
9b9f6cd2f9 some fixes maybe 2024-08-27 15:07:18 +00:00
rootdarkarchon
9d9573a510 try locking metrics 2024-08-27 15:07:18 +00:00
rootdarkarchon
a42f553648 make cleanup check time configurable, add distribution file server address
handle download errors on shards better

flush filestream

oml
2024-08-27 14:46:36 +00:00
rootdarkarchon
7c6dbafce9 set user agent correctly
add statistics

fix bug
2024-08-27 14:46:34 +00:00
rootdarkarchon
d54d793575 refactor file server 2024-08-27 14:31:41 +00:00
rootdarkarchon
d40730bca3 some file server fixes I guess
break, not return

fix queue processing

fix queue processing, again

fix queue processing, the third

do not use async for queue processing

something
2024-08-27 14:22:10 +00:00
Loporrit
d1fcc87f37 Disable priority queue again 2024-08-27 14:21:04 +00:00
rootdarkarchon
e74f511cc0 add alias to jwt, remove caching from auth, remove db usage from files 2024-08-27 14:11:33 +00:00
rootdarkarchon
989046da13 add geoip service for file shard matching
fix bug

more logging

fix logging wtf is going on even

handle lists in config log output

do not set "*" as default continent

do not rely on "*" being present in configuration when picking file shard
2024-08-27 14:11:23 +00:00
rootdarkarchon
7667b74734 bugfix 2024-08-27 13:42:03 +00:00
rootdarkarchon
566634776b clean up downloads on shards 2024-08-27 13:41:08 +00:00
rootdarkarchon
9eec562fdd use maredbcontext from controller 2024-08-27 13:28:18 +00:00
Loporrit
7c95497e33 Add .pbd 2024-08-24 05:47:45 +00:00
Loporrit
c7705dde99 fix /message 2024-08-24 05:47:33 +00:00
Stanley Dimant
348781e5f5 remove grpc 2024-06-11 07:31:34 +00:00
Stanley Dimant
72df676c9d update server to net 8
up conn semaphore

add http logging to files

update prometheus?

downgrade again to net7?

temp downgrade to net7 again?

net8 for static file server attempt #2

loporrit: MareAPI .NET8 update
2024-06-11 07:31:20 +00:00
rootdarkarchon
8121b8c1f6 fix upload getting stuck
fix disposed semaphores
2024-03-14 19:06:58 +00:00
Loporrit
be1edba5c2 Remove group bans on user deletion 2024-03-14 18:50:55 +00:00
rootdarkarchon
b2e4b99b10 use maredbcontext from controller 2024-03-14 18:39:08 +00:00
Loporrit
3cfe6fbdee Disable high priority queue 2024-03-14 18:37:52 +00:00
rootdarkarchon
701a5763b5 add high priority queue 2024-03-14 18:36:37 +00:00
rootdarkarchon
3b77a179c5 expire requests regardless of queue presence of entries 2024-03-14 18:35:53 +00:00
Loporrit
03025165be Quiet profile reports 2024-03-14 18:34:57 +00:00
Loporrit
d4f017e16e Disable more commands and vanity cleanup 2024-03-14 18:34:47 +00:00