Author SHA1 Message Date
static 48bf9c1165 사소한 리팩토링 2026-10-20 03:12:53 +09:00
static d986493431 업로드 및 썸네일 등록의 경쟁 조건을 수정하고 캐시 무효화와 스트리밍 캐시의 메모리 사용량을 개선 2026-10-12 14:02:41 +09:00
staticandClaude Opus 5.5 75e8ed6c77 파일 시스템 및 업로드 관련 비즈니스 규칙과 트랜잭션 경계를 Repository에서 서비스로 이동하고 IntegrityError를 제거하며 즐겨찾기 API를 멱등하게 변경
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 12:13:58 +09:00
staticandClaude Opus 5.5 73b2d29b2c DB 접근 시 Executor를 명시적으로 전달하고 서비스에서 트랜잭션 경계를 관리하도록 인증 및 키 관련 코드를 개선하고 직렬화 실패 시 트랜잭션을 재시도하도록 변경
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 12:05:34 +09:00
staticandClaude Opus 5.5 5b8621e524 리다이렉트 경로 처리, 파일 정보 변환, 상태 표시 UI 등 중복된 코드를 통합하고 모듈 간 순환 의존성 및 계층 위반 해소
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 08:51:07 +09:00
staticandClaude Opus 5.5 9119c21881 형식이 잘못된 세션 쿠키로 인해 서버 오류가 발생하던 문제와 업로드 중 오류가 무시되거나 무한히 대기하던 문제 등 버그 수정
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 08:19:41 +09:00
staticandClaude Opus 5.5 bb0328b39e Playwright 기반 E2E 테스트 추가
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:42:51 +09:00
staticandClaude Opus 5.5 306fcb2420 Svelte 컴포넌트를 제외한 파일, 폴더 및 라우팅 경로의 이름을 kebab-case로 통일하고 import 경로에 실제 파일 확장자를 사용하도록 변경
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 07:41:28 +09:00
staticandClaude Opus 5.5 225ad3515f 파일을 업로드할 때 캐시에 저장하고 동영상을 스트리밍하는 과정에서도 청크 단위로 캐시에 저장하여 다음 재생에 활용하도록 개선
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 01:40:03 +09:00
staticandClaude Opus 5.5 06fdfcc447 스트리밍되는 동영상의 썸네일을 생성할 때 프레임이 준비되지 않으면 무한히 대기하던 문제 수정
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 01:00:23 +09:00
staticandClaude Opus 5.5 3e52fa7a7c 파일을 다운로드할 때 전체 파일을 메모리에 올리는 대신 서비스 워커에서 스트리밍으로 복호화하도록 변경
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:48:21 +09:00
staticandClaude Opus 5.5 877cf9c42e 컴포넌트를 도메인과 무관한 UI 컴포넌트와 도메인 컴포넌트로 재구성하고 중복된 파일 목록 컴포넌트 통합 및 모듈 간 의존성 방향을 ESLint로 강제
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:30:10 +09:00
staticandClaude Opus 5.5 4e82c5e556 파일 시스템 관련 상태와 변경 로직을 도메인 모듈로 통합하여 캐시가 자동으로 갱신되도록 개선하고 오류를 토스트로 표시하도록 변경
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:22:25 +09:00
staticandClaude Opus 5.5 2ce73aa331 키 관리 로직을 Keyring으로 통합하여 하드코딩된 MEK 및 HSK 버전을 제거하고 Svelte Store를 Rune으로 대체
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:10:37 +09:00
staticandClaude Opus 5.5 aee87fd9b6 서버, 클라이언트, 서비스 워커 공용 코드를 lib/shared로 분리하고 브라우저 전용 모듈을 lib/client로 재구성
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:04:56 +09:00
staticandClaude Opus 5.5 6a6aa1765b 서버 코드를 Router, Service, Repository 계층으로 분리하고 오류 처리 및 파일 삭제 로직 개선
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 00:00:21 +09:00
staticandClaude Opus 5.5 ed732be999 상위 디렉터리 및 카테고리의 소유권을 검증하지 않던 문제 등 버그 수정 및 Vitest 기반 테스트 추가
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:45:56 +09:00
staticandClaude Opus 5.5 67989f4741 패키지 버전 업데이트 및 SvelteKit 3, Tailwind CSS 4로 마이그레이션
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:49:24 +09:00
staticandClaude Opus 5.5 35f3e92b5a SSR을 비활성화하여 SPA로 전환
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:49:24 +09:00
static b35585c7bb 패키지 버전 업데이트 2026-07-15 19:27:24 +09:00
static 7f68f6d580 파일을 업로드할 때 청크별로 개별 저장하는 대신 파일 하나에 저장하도록 변경 2026-03-10 22:44:11 +09:00
static c2874035ba 암호화 마이그레이션 기능 삭제 2026-03-10 19:33:12 +09:00
static 3c1e98c872 패키지 버전 업데이트 2026-03-09 06:14:37 +09:00
static 9635d2a51b IndexedDB에 가끔 파일 및 디렉터리 정보를 저장하지 못하던 버그 수정 2026-01-18 16:32:06 +09:00
static 3b0cfd5a92 즐겨찾기 검색 필터를 재귀적으로 동작하도록 변경 2026-01-18 16:16:38 +09:00
static 2f6d35c335 모바일 환경에서 SearchBar의 레이아웃이 깨지는 문제 수정 2026-01-18 14:16:40 +09:00
static 72babc532f 검색 필터에 즐겨찾기 여부 추가 2026-01-18 13:29:06 +09:00
static 63163d6279 파일 및 디렉터리 메타데이터 복호화 로직 리팩토링 2026-01-18 13:01:44 +09:00
static 4797ccfd23 FileRepo의 함수 중 디렉터리 관련된 함수들을 DirectoryRepo로 분리 2026-01-18 12:34:04 +09:00
static 14693160b8 사소한 리팩토링 2026-01-18 12:03:24 +09:00
static bcb57bb12d IndexedDB에 즐겨찾기 여부를 항상 저장하도록 변경 2026-01-18 11:33:30 +09:00
static ff6ea3a0b9 파일 페이지에서 즐겨찾기 설정이 가능하도록 변경 및 즐겨찾기에 추가된 경우 목록에서 즐겨찾기 여부를 아이콘으로 표시하도록 개선 2026-01-17 20:11:01 +09:00
static 420e30f677 즐겨찾기 기능 구현 2026-01-17 19:41:52 +09:00
static befa535526 비효율적인 비디오 스트리밍 로직 개선 2026-01-17 18:25:54 +09:00
static d8e18fb1d3 OR 조건 대신 IN 연산자를 사용하도록 일부 SQL 쿼리 수정 2026-01-16 10:24:32 +09:00
static cb5105a355 패키지 버전 업데이트 2026-01-16 07:51:17 +09:00
static fe83a71a1f 썸네일이 누락된 파일 조회 및 레거시 파일 조회 네트워크 호출 최적화 2026-01-15 20:33:27 +09:00
static ebcdbd2d83 이름 검색 로직 개선 및 뒤로가기로 검색 페이지로 돌아온 경우에 필터 및 검색 결과가 유지되도록 개선 2026-01-15 18:25:01 +09:00
static b3e3671c09 파일 검색 쿼리 최적화 2026-01-15 15:54:45 +09:00
static 37bd6a9315 검색 기능 구현 2026-01-15 15:11:03 +09:00
static 96d5397cb5 docker.yaml 파일의 오타 수정 2026-01-13 00:37:29 +09:00
static b952bfae86 릴리즈 때마다 Docker 이미지를 자동으로 빌드하는 Action 추가 2026-01-13 00:28:52 +09:00
static 4cdf2b342f 청크 업로드 성능 개선 및 네트워크 속도를 더 정확하게 측정하도록 개선 2026-01-12 23:37:04 +09:00
static a4912c8952 사소한 리팩토링 2026-01-12 20:50:19 +09:00
static 00b9858db7 업로드된 청크 목록을 비트맵을 활용해 효율적으로 저장하도록 개선 2026-01-12 18:37:36 +09:00
static c778a4fb9e 파일 업로드 로직 리팩토링 2 2026-01-12 16:58:28 +09:00
static e7dc96bb47 HMAC 계산을 Web Worker에서 처리하도록 변경 2026-01-12 15:16:43 +09:00
static b636d75ea0 파일 업로드 로직 리팩토링 2026-01-12 12:02:20 +09:00
static 27e90ef4d7 이전 버전에서 업로드된 파일을 청크 업로드 방식으로 마이그레이션할 수 있는 기능 추가 2026-01-12 08:40:07 +09:00
static 594c3654c9 파일 및 썸네일 다운로드 Endpoint의 핸들러를 하나로 통합 2026-01-12 05:04:07 +09:00
static 614d0e74b4 패키지 버전 업데이트 2026-01-11 16:01:02 +09:00
static efc2b08b1f Merge pull request #18 from kmc7468/add-chunked-upload
Chunked Upload 도입
2026-01-11 15:56:35 +09:00
static 80368c3a29 사소한 리팩토링 2 2026-01-11 15:54:05 +09:00
static 83369f83e3 DB에 청크 업로드 경로를 저장하도록 변경 2026-01-11 15:16:03 +09:00
static 2801eed556 사소한 리팩토링 2026-01-11 14:35:30 +09:00
static 57c27b76be 썸네일 업로드도 새로운 업로드 방식으로 변경 2026-01-11 14:07:32 +09:00
static 3628e6d21a 업로드할 때에도 스트리밍 방식으로 처리하도록 변경 2026-01-11 13:19:54 +09:00
394 changed files with 14807 additions and 8875 deletions

No files matched your search

+3 -1
View File
@@ -12,7 +12,6 @@ node_modules
/data /data
/library /library
/thumbnails /thumbnails
/uploads
# OS # OS
.DS_Store .DS_Store
@@ -31,3 +30,6 @@ Thumbs.db
# Vite # Vite
vite.config.js.timestamp-* vite.config.js.timestamp-*
vite.config.ts.timestamp-* vite.config.ts.timestamp-*
# E2E tests
/test/e2e/results
-1
View File
@@ -12,4 +12,3 @@ USER_CLIENT_CHALLENGE_EXPIRES=
SESSION_UPGRADE_CHALLENGE_EXPIRES= SESSION_UPGRADE_CHALLENGE_EXPIRES=
LIBRARY_PATH= LIBRARY_PATH=
THUMBNAILS_PATH= THUMBNAILS_PATH=
UPLOADS_PATH=
+45
View File
@@ -0,0 +1,45 @@
name: Docker Image Build
on:
release:
types: [published]
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract Docker metadata
uses: docker/metadata-action@v5
id: meta
with:
images: ghcr.io/${{ github.repository }}
tags: |
type=semver,pattern={{version}}
type=raw,value=latest
type=sha
- name: Build and push Docker image
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
+3 -1
View File
@@ -10,7 +10,6 @@ node_modules
/data /data
/library /library
/thumbnails /thumbnails
/uploads
# OS # OS
.DS_Store .DS_Store
@@ -29,3 +28,6 @@ Thumbs.db
# Vite # Vite
vite.config.js.timestamp-* vite.config.js.timestamp-*
vite.config.ts.timestamp-* vite.config.ts.timestamp-*
# E2E tests
/test/e2e/results
+1
View File
@@ -1,6 +1,7 @@
{ {
"printWidth": 100, "printWidth": 100,
"plugins": ["prettier-plugin-svelte", "prettier-plugin-tailwindcss"], "plugins": ["prettier-plugin-svelte", "prettier-plugin-tailwindcss"],
"tailwindStylesheet": "./src/app.css",
"overrides": [ "overrides": [
{ {
"files": "*.svelte", "files": "*.svelte",
+2 -2
View File
@@ -2,8 +2,8 @@
FROM node:22-alpine AS base FROM node:22-alpine AS base
WORKDIR /app WORKDIR /app
RUN npm install -g pnpm@10 RUN npm install -g pnpm@12
COPY pnpm-lock.yaml . COPY pnpm-lock.yaml pnpm-workspace.yaml ./
# Build Stage # Build Stage
FROM base AS build FROM base AS build
-1
View File
@@ -20,7 +20,6 @@ services:
- SESSION_UPGRADE_CHALLENGE_EXPIRES - SESSION_UPGRADE_CHALLENGE_EXPIRES
- LIBRARY_PATH=/app/data/library - LIBRARY_PATH=/app/data/library
- THUMBNAILS_PATH=/app/data/thumbnails - THUMBNAILS_PATH=/app/data/thumbnails
- UPLOADS_PATH=/app/data/uploads
# SvelteKit # SvelteKit
- ADDRESS_HEADER=${TRUST_PROXY:+X-Forwarded-For} - ADDRESS_HEADER=${TRUST_PROXY:+X-Forwarded-For}
- XFF_DEPTH=${TRUST_PROXY:-} - XFF_DEPTH=${TRUST_PROXY:-}
+35 -2
View File
@@ -1,5 +1,6 @@
import { includeIgnoreFile } from "@eslint/compat"; import { includeIgnoreFile } from "@eslint/compat";
import js from "@eslint/js"; import js from "@eslint/js";
import { loadConfig } from "@sveltejs/load-config";
import { defineConfig } from "eslint/config"; import { defineConfig } from "eslint/config";
import prettier from "eslint-config-prettier"; import prettier from "eslint-config-prettier";
import svelte from "eslint-plugin-svelte"; import svelte from "eslint-plugin-svelte";
@@ -7,16 +8,31 @@ import tailwind from "eslint-plugin-tailwindcss";
import globals from "globals"; import globals from "globals";
import ts from "typescript-eslint"; import ts from "typescript-eslint";
import { fileURLToPath } from "url"; import { fileURLToPath } from "url";
import svelteConfig from "./svelte.config.js";
const svelteConfig = (await loadConfig("./", { traverse: false }))?.config;
const gitignorePath = fileURLToPath(new URL("./.gitignore", import.meta.url)); const gitignorePath = fileURLToPath(new URL("./.gitignore", import.meta.url));
// Restricts the direction of dependencies between modules
const defineImportBoundaries = (boundaries) =>
Object.entries(boundaries).map(([files, regex]) => ({
files: [files],
rules: {
"no-restricted-imports": [
"error",
{
patterns: [{ regex, message: "Violates the dependency direction between modules." }],
},
],
},
}));
export default defineConfig( export default defineConfig(
includeIgnoreFile(gitignorePath), includeIgnoreFile(gitignorePath),
js.configs.recommended, js.configs.recommended,
...ts.configs.recommended, ...ts.configs.recommended,
...svelte.configs.recommended, ...svelte.configs.recommended,
...tailwind.configs["flat/recommended"], tailwind.configs.recommended,
prettier, prettier,
...svelte.configs.prettier, ...svelte.configs.prettier,
{ {
@@ -26,10 +42,27 @@ export default defineConfig(
...globals.node, ...globals.node,
}, },
}, },
settings: {
tailwindcss: {
cssConfigPath: "./src/app.css",
},
},
rules: { rules: {
"no-undef": "off", "no-undef": "off",
}, },
}, },
...defineImportBoundaries({
// Pure code shared by the server, the client, and the service worker
"src/lib/shared/**": "^(#lib/(?!shared/)|#trpc/|#workers/|\\$app/|svelte($|/))",
"src/lib/server/**": "^(#lib/(client|features|ui)/|#trpc/)",
// Infrastructure of the browser
"src/lib/client/**": "^#lib/(server|features|ui)/",
// Domain-agnostic components
"src/lib/ui/**": "^(#lib/(server|client|features)/|#trpc/)",
"src/lib/features/**": "^#lib/server/",
"src/service-worker/**":
"^(#lib/(?!shared/|client/storage/|client/cache/file-cache\\.ts$)|#trpc/|#workers/)",
}),
{ {
files: ["**/*.svelte", "**/*.svelte.ts", "**/*.svelte.js"], files: ["**/*.svelte", "**/*.svelte.ts", "**/*.svelte.js"],
languageOptions: { languageOptions: {
+11 -11
View File
@@ -1,17 +1,17 @@
import { defineConfig } from "kysely-ctl"; import { defineConfig } from "kysely-ctl";
import { Pool } from "pg"; import { createDialect } from "./src/lib/server/db/dialect.ts";
// Empty values (e.g. `DATABASE_HOST=` in .env) are treated as unset, as in src/env.ts
const optional = (name: string) => process.env[name] || undefined;
export default defineConfig({ export default defineConfig({
dialect: "pg", dialect: createDialect({
dialectConfig: { host: optional("DATABASE_HOST"),
pool: new Pool({ port: optional("DATABASE_PORT") ? Number(optional("DATABASE_PORT")) : undefined,
host: process.env.DATABASE_HOST, user: optional("DATABASE_USER"),
port: process.env.DATABASE_PORT ? parseInt(process.env.DATABASE_PORT) : undefined, password: optional("DATABASE_PASSWORD"),
user: process.env.DATABASE_USER, name: optional("DATABASE_NAME"),
password: process.env.DATABASE_PASSWORD, }),
database: process.env.DATABASE_NAME,
}),
},
migrations: { migrations: {
migrationFolder: "./src/lib/server/db/migrations", migrationFolder: "./src/lib/server/db/migrations",
}, },
+55 -42
View File
@@ -1,73 +1,86 @@
{ {
"name": "arkvault", "name": "arkvault",
"private": true, "private": true,
"version": "0.7.0", "version": "0.9.1",
"type": "module", "type": "module",
"scripts": { "scripts": {
"dev": "vite dev", "dev": "vite dev",
"dev:remote": "__VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS=.ts.net vite dev --host 0.0.0.0",
"build": "vite build", "build": "vite build",
"preview": "vite preview", "preview": "vite preview",
"check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json", "check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json && tsc --noEmit --project ./src/service-worker",
"check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch", "check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch",
"format": "prettier --write .", "format": "prettier --write .",
"lint": "prettier --check . && eslint .", "lint": "prettier --check . && eslint .",
"test": "vitest run",
"test:watch": "vitest",
"test:e2e": "bash test/e2e/run.sh",
"db:up": "docker compose -f docker-compose.dev.yaml -p arkvault-dev up -d", "db:up": "docker compose -f docker-compose.dev.yaml -p arkvault-dev up -d",
"db:down": "docker compose -f docker-compose.dev.yaml -p arkvault-dev down", "db:down": "docker compose -f docker-compose.dev.yaml -p arkvault-dev down",
"db:migrate": "kysely migrate" "db:migrate": "kysely migrate"
}, },
"devDependencies": { "devDependencies": {
"@eslint/compat": "^2.0.0", "@eslint/compat": "^2.1.1",
"@eslint/js": "^9.39.2", "@eslint/js": "^10.0.1",
"@iconify-json/material-symbols": "^1.2.50", "@iconify-json/material-symbols": "^1.2.95",
"@sveltejs/adapter-node": "^5.4.0", "@noble/hashes": "^2.4.0",
"@sveltejs/kit": "^2.49.2", "@playwright/test": "1.64.0",
"@sveltejs/vite-plugin-svelte": "^6.2.1", "@sveltejs/adapter-node": "^6.0.0",
"@tanstack/svelte-virtual": "^3.13.16", "@sveltejs/kit": "^3.0.1",
"@trpc/client": "^11.8.1", "@sveltejs/load-config": "^0.2.2",
"@sveltejs/vite-plugin-svelte": "^7.3.1",
"@tailwindcss/vite": "^4.3.3",
"@tanstack/svelte-virtual": "^3.13.40",
"@testcontainers/postgresql": "^12.2.0",
"@trpc/client": "^11.19.0",
"@types/file-saver": "^2.0.7", "@types/file-saver": "^2.0.7",
"@types/ms": "^0.7.34", "@types/ms": "^2.1.0",
"@types/node-schedule": "^2.1.8", "@types/node-schedule": "^2.1.8",
"@types/pg": "^8.16.0", "@types/pg": "^8.23.1",
"autoprefixer": "^10.4.23", "axios": "^1.20.0",
"axios": "^1.13.2", "dexie": "^4.4.6",
"dexie": "^4.2.1", "es-hangul": "^2.4.0",
"eslint": "^9.39.2", "eslint": "^10.12.0",
"eslint-config-prettier": "^10.1.8", "eslint-config-prettier": "^10.1.8",
"eslint-plugin-svelte": "^3.13.1", "eslint-plugin-svelte": "^3.23.1",
"eslint-plugin-tailwindcss": "^3.18.2", "eslint-plugin-tailwindcss": "^4.4.0",
"exifreader": "^4.33.1", "exifreader": "^4.48.0",
"file-saver": "^2.0.5", "file-saver": "^2.0.5",
"globals": "^16.5.0", "globals": "^17.13.0",
"heic2any": "^0.0.4", "heic2any": "^0.0.4",
"kysely-ctl": "^0.19.0", "kysely-ctl": "^0.21.0",
"lru-cache": "^11.2.4", "lru-cache": "^11.5.3",
"mime": "^4.1.0", "mime": "^4.1.0",
"p-limit": "^7.2.0", "p-limit": "^7.3.3",
"prettier": "^3.7.4", "prettier": "^3.9.10",
"prettier-plugin-svelte": "^3.4.1", "prettier-plugin-svelte": "^4.1.1",
"prettier-plugin-tailwindcss": "^0.7.2", "prettier-plugin-tailwindcss": "^0.8.1",
"svelte": "^5.46.1", "svelte": "^5.57.2",
"svelte-check": "^4.3.5", "svelte-check": "^4.7.6",
"tailwindcss": "^3.4.19", "tailwindcss": "^4.3.3",
"typescript": "^5.9.3", "typescript": "^6.0.3",
"typescript-eslint": "^8.51.0", "typescript-eslint": "^8.71.1",
"unplugin-icons": "^22.5.0", "unplugin-icons": "^24.0.0",
"vite": "^7.3.0" "vite": "^8.3.4",
"vitest": "^5.0.3"
}, },
"dependencies": { "dependencies": {
"@fastify/busboy": "^3.2.0", "@trpc/server": "^11.19.0",
"@trpc/server": "^11.8.1", "argon2": "^0.45.1",
"argon2": "^0.44.0", "kysely": "^0.29.6",
"kysely": "^0.28.9",
"ms": "^2.1.3", "ms": "^2.1.3",
"node-schedule": "^2.1.1", "node-schedule": "^2.1.1",
"pg": "^8.16.3", "pg": "^8.23.1",
"superjson": "^2.2.6", "superjson": "^2.2.6",
"uuid": "^13.0.0", "zod": "^4.6.5"
"zod": "^4.3.5"
}, },
"engines": { "engines": {
"node": "^22.0.0", "node": "^22.17.0",
"pnpm": "^10.0.0" "pnpm": "^12.0.0"
},
"imports": {
"#lib/*": "./src/lib/*",
"#trpc/*": "./src/trpc/*",
"#workers/*": "./src/workers/*"
} }
} }
+2964 -1785
View File
File diff suppressed because it is too large. Load diff
+6
View File
@@ -0,0 +1,6 @@
allowBuilds:
argon2: true
cpu-features: false
exifreader: false
protobufjs: false
ssh2: false
-6
View File
@@ -1,6 +0,0 @@
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
};
+48 -3
View File
@@ -1,3 +1,48 @@
@import "tailwindcss/base"; @import "tailwindcss";
@import "tailwindcss/components";
@import "tailwindcss/utilities"; @theme {
--color-primary-100: #ffe3e3;
--color-primary-200: #ffc9c9;
--color-primary-300: #ffa8a8;
--color-primary-400: #ff8787;
--color-primary-500: #ff6b6b;
--color-primary-600: #fa5252;
--color-primary-700: #f03e3e;
--color-primary-800: #e03131;
--color-primary-900: #c92a2a;
--color-primary-000: #fff5f5;
--color-gray-100: #f1f3f5;
--color-gray-200: #e9ecef;
--color-gray-300: #dee2e6;
--color-gray-400: #ced4da;
--color-gray-500: #adb5bd;
--color-gray-600: #868e96;
--color-gray-700: #495057;
--color-gray-800: #343a40;
--color-gray-900: #212529;
--color-gray-000: #f8f9fa;
--font-sans:
"Pretendard Variable", Pretendard, -apple-system, BlinkMacSystemFont, system-ui, Roboto,
"Helvetica Neue", "Segoe UI", "Apple SD Gothic Neo", "Noto Sans KR", "Malgun Gothic",
"Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", sans-serif;
}
/*
The default border color has changed to `currentcolor` in Tailwind CSS v4,
so we've added these compatibility styles to make sure everything still
looks the same as it did with Tailwind CSS v3.
If we ever want to remove these styles, we need to add an explicit border
color utility to any element that depends on these defaults.
*/
@layer base {
*,
::after,
::before,
::backdrop,
::file-selector-button {
border-color: var(--color-gray-200, currentcolor);
}
}
+45
View File
@@ -0,0 +1,45 @@
import { building } from "$app/env";
import { defineEnvVars } from "@sveltejs/kit/env";
import ms, { type StringValue } from "ms";
// Empty values (e.g. `LIBRARY_PATH=` in .env) are treated as unset
const required = (value: string | undefined) => {
if (!value && !building) throw new Error("Required environment variable not set");
return value ?? "";
};
const optional = (value: string | undefined) => value || undefined;
const withDefault = (defaultValue: string) => (value: string | undefined) => value || defaultValue;
const duration = (defaultValue: StringValue) => (value: string | undefined) => {
const result = ms((value || defaultValue) as StringValue);
if (!Number.isFinite(result)) throw new Error(`Invalid duration: ${value}`);
return result;
};
export const variables = defineEnvVars({
NODE_ENV: { schema: withDefault("development") },
DATABASE_HOST: { schema: optional },
DATABASE_PORT: {
schema: (value) => {
if (!value) return undefined;
const port = Number(value);
if (!Number.isInteger(port)) throw new Error(`Invalid port: ${value}`);
return port;
},
},
DATABASE_USER: { schema: optional },
DATABASE_PASSWORD: { schema: required },
DATABASE_NAME: { schema: optional },
SESSION_SECRET: { schema: required },
SESSION_EXPIRES: { schema: duration("14d") },
USER_CLIENT_CHALLENGE_EXPIRES: { schema: duration("5m") },
SESSION_UPGRADE_CHALLENGE_EXPIRES: { schema: duration("5m") },
LIBRARY_PATH: { schema: withDefault("library") },
THUMBNAILS_PATH: { schema: withDefault("thumbnails") },
});
+9 -37
View File
@@ -1,7 +1,8 @@
import type { ClientInit } from "@sveltejs/kit"; import type { ClientInit } from "@sveltejs/kit/hooks";
import { cleanupDanglingInfos, getClientKey, getMasterKeys, getHmacSecrets } from "$lib/indexedDB"; import { cleanupFileCaches } from "#lib/client/cache/index.ts";
import { prepareFileCache } from "$lib/modules/file"; import { keyring } from "#lib/client/keys/index.ts";
import { clientKeyStore, masterKeyStore, hmacSecretStore } from "$lib/stores"; import { listenForServiceWorkerMessages } from "#lib/client/service-worker.ts";
import { cleanupDanglingInfos } from "#lib/client/storage/indexed-db/index.ts";
const requestPersistentStorage = async () => { const requestPersistentStorage = async () => {
const isPersistent = await navigator.storage.persist(); const isPersistent = await navigator.storage.persist();
@@ -12,40 +13,11 @@ const requestPersistentStorage = async () => {
} }
}; };
const prepareClientKeyStore = async () => {
const [encryptKey, decryptKey, signKey, verifyKey] = await Promise.all([
getClientKey("encrypt"),
getClientKey("decrypt"),
getClientKey("sign"),
getClientKey("verify"),
]);
if (encryptKey && decryptKey && signKey && verifyKey) {
clientKeyStore.set({ encryptKey, decryptKey, signKey, verifyKey });
}
};
const prepareMasterKeyStore = async () => {
const masterKeys = await getMasterKeys();
if (masterKeys.length > 0) {
masterKeyStore.set(new Map(masterKeys.map((masterKey) => [masterKey.version, masterKey])));
}
};
const prepareHmacSecretStore = async () => {
const hmacSecrets = await getHmacSecrets();
if (hmacSecrets.length > 0) {
hmacSecretStore.set(new Map(hmacSecrets.map((hmacSecret) => [hmacSecret.version, hmacSecret])));
}
};
export const init: ClientInit = async () => { export const init: ClientInit = async () => {
await Promise.all([ listenForServiceWorkerMessages();
requestPersistentStorage(),
prepareFileCache(), await Promise.all([requestPersistentStorage(), keyring.load()]);
prepareClientKeyStore(),
prepareMasterKeyStore(),
prepareHmacSecretStore(),
]);
cleanupDanglingInfos(); // Intended cleanupDanglingInfos(); // Intended
cleanupFileCaches().catch(console.error);
}; };
+20 -14
View File
@@ -1,24 +1,30 @@
import type { ServerInit } from "@sveltejs/kit"; import { sequence, type ServerInit } from "@sveltejs/kit/hooks";
import { sequence } from "@sveltejs/kit/hooks";
import schedule from "node-schedule"; import schedule from "node-schedule";
import { cleanupExpiredUserClientChallenges } from "$lib/server/db/client"; import { migrateDB } from "#lib/server/db/kysely.ts";
import { migrateDB } from "$lib/server/db/kysely"; import { authenticate, setAgentInfo } from "#lib/server/middlewares/index.ts";
import { import {
cleanupExpiredSessions, AuthService,
cleanupExpiredSessionUpgradeChallenges, ClientService,
} from "$lib/server/db/session"; FileDeletionService,
import { cleanupExpiredUploadSessions } from "$lib/server/db/upload"; UploadService,
import { authenticate, setAgentInfo } from "$lib/server/middlewares"; } from "#lib/server/services/index.ts";
export const init: ServerInit = async () => { export const init: ServerInit = async () => {
await migrateDB(); await migrateDB();
schedule.scheduleJob("0 * * * *", () => { schedule.scheduleJob("0 * * * *", async () => {
cleanupExpiredUserClientChallenges(); // Failures are logged and retried at the next run, so they don't crash the server
cleanupExpiredSessions(); const results = await Promise.allSettled([
cleanupExpiredSessionUpgradeChallenges(); AuthService.cleanupExpiredSessions(),
cleanupExpiredUploadSessions(); ClientService.cleanupExpiredChallenges(),
UploadService.cleanupExpiredUploadSessions(),
]);
results.forEach((result) => {
if (result.status === "rejected") console.error(result.reason);
});
FileDeletionService.schedulePendingFileDeletions();
}); });
FileDeletionService.schedulePendingFileDeletions();
}; };
export const handle = sequence(setAgentInfo, authenticate); export const handle = sequence(setAgentInfo, authenticate);
+185
View File
@@ -0,0 +1,185 @@
import {
deleteFileCacheIndex,
getFileCacheIndex as getAllFileCacheIndexes,
getFileCacheIndexOf,
storeFileCacheIndex,
updateFileCacheRetrievedAt,
} from "#lib/client/storage/indexed-db/cache-index.ts";
import {
createWritable,
deleteDirectory,
deleteFile,
deleteFilesInDirectory,
getFile,
listDirectoryNames,
listFileNames,
moveFile,
} from "#lib/client/storage/opfs.ts";
import { CHUNK_SIZE, getChunkCount } from "#lib/shared/format/index.ts";
// Caches the plaintext of files in chunks, so that partially loaded files (e.g. videos which were
// partially played) can also be cached. Chunk `i` starts at `i * CHUNK_SIZE` of the plaintext,
// except that legacy files are cached as a single chunk. Used by both the pages and the service
// worker, so the index is always read from IndexedDB.
export interface FileCacheLayout {
totalSize: number;
chunkCount: number;
}
const getDirectoryPath = (fileId: number) => `/cache/file/${fileId}`;
const getChunkPath = (fileId: number, chunkIndex: number) =>
`${getDirectoryPath(fileId)}/${chunkIndex}`;
const getChunkSize = ({ totalSize, chunkCount }: FileCacheLayout, chunkIndex: number) => {
return chunkCount === 1 ? totalSize : Math.min(CHUNK_SIZE, totalSize - chunkIndex * CHUNK_SIZE);
};
export const getFileCacheIndex = async () => {
return await getAllFileCacheIndexes();
};
export const getCachedChunkIndexes = async (fileId: number) => {
const names = await listFileNames(getDirectoryPath(fileId));
return new Set(names.filter((name) => /^\d+$/.test(name)).map(Number));
};
export const readCachedChunk = async (fileId: number, chunkIndex: number) => {
return await getFile(getChunkPath(fileId, chunkIndex));
};
export const touchFileCache = async (fileId: number) => {
await updateFileCacheRetrievedAt(fileId, new Date());
};
const updateIndex = async (fileId: number, layout: FileCacheLayout) => {
const chunkIndexes = await getCachedChunkIndexes(fileId);
const size = chunkIndexes
.values()
.filter((chunkIndex) => chunkIndex < layout.chunkCount)
.reduce((size, chunkIndex) => size + getChunkSize(layout, chunkIndex), 0);
const now = new Date();
await storeFileCacheIndex({
fileId,
cachedAt: (await getFileCacheIndexOf(fileId))?.cachedAt ?? now,
lastRetrievedAt: now,
size,
totalSize: layout.totalSize,
chunkCount: layout.chunkCount,
isComplete: size === layout.totalSize,
});
};
// Writes of the same file are serialized so that the index isn't updated with a stale state
const writeQueues = new Map<number, Promise<unknown>>();
const enqueueWrite = <T>(fileId: number, write: () => Promise<T>) => {
const result = (writeQueues.get(fileId) ?? Promise.resolve()).catch(() => {}).then(write);
const settled = result.catch(() => {});
writeQueues.set(fileId, settled);
void settled.then(() => {
if (writeQueues.get(fileId) === settled) {
writeQueues.delete(fileId);
}
});
return result;
};
export const storeCachedChunk = (
fileId: number,
chunkIndex: number,
data: Uint8Array | Blob,
layout: FileCacheLayout,
) => enqueueWrite(fileId, () => writeChunk(fileId, chunkIndex, data, layout));
const writeChunk = async (
fileId: number,
chunkIndex: number,
data: Uint8Array | Blob,
layout: FileCacheLayout,
) => {
const path = getChunkPath(fileId, chunkIndex);
const tempPath = `${path}.${crypto.randomUUID()}.tmp`;
// Written to a temporary file first, so that incomplete chunks are never visible
const writable = await createWritable(tempPath);
try {
await writable.write(data as FileSystemWriteChunkType);
await writable.close();
await moveFile(tempPath, path);
} catch (e) {
await writable.abort().catch(() => {});
await deleteFile(tempPath).catch(() => {});
throw e;
}
await updateIndex(fileId, layout);
};
const UPLOAD_CACHE_DIRECTORY = "/cache/upload";
const STALE_UPLOAD_CACHE_AGE = 24 * 3600 * 1000; // 24 hours
// Caches the plaintext of a file while it is being uploaded. Chunks are written to a temporary
// directory, and moved to the cache once the id of the file is known.
export const createUploadFileCache = (totalSize: number) => {
const layout = { totalSize, chunkCount: getChunkCount(totalSize) };
// Prefixed with the creation time to find stale ones (e.g. the tab was closed while uploading)
const directoryPath = `${UPLOAD_CACHE_DIRECTORY}/${Date.now()}-${crypto.randomUUID()}`;
const writtenChunkIndexes = new Set<number>();
let isFailed = false;
const discard = async () => {
isFailed = true;
await deleteDirectory(directoryPath).catch(() => {});
};
return {
// Failing to cache shouldn't fail the upload, so errors are ignored
async writeChunk(chunkIndex: number, data: ArrayBuffer) {
if (isFailed) return;
try {
const writable = await createWritable(`${directoryPath}/${chunkIndex}`);
await writable.write(data);
await writable.close();
writtenChunkIndexes.add(chunkIndex);
} catch (e) {
console.error(e);
await discard();
}
},
async commit(fileId: number) {
if (isFailed || writtenChunkIndexes.size !== layout.chunkCount) {
await discard();
return;
}
await enqueueWrite(fileId, async () => {
for (const chunkIndex of writtenChunkIndexes) {
await moveFile(`${directoryPath}/${chunkIndex}`, getChunkPath(fileId, chunkIndex));
}
await updateIndex(fileId, layout);
}).finally(discard);
},
discard,
};
};
export const deleteFileCache = async (fileId: number) => {
await deleteFileCacheIndex(fileId);
await deleteDirectory(getDirectoryPath(fileId));
};
export const cleanupFileCaches = async () => {
// Files were cached as single files directly under `/cache` before
await deleteFilesInDirectory("/cache");
// Uploads might be in progress in other tabs, so only stale ones are deleted
for (const name of await listDirectoryNames(UPLOAD_CACHE_DIRECTORY)) {
const createdAt = parseInt(name, 10);
if (!(Date.now() - createdAt < STALE_UPLOAD_CACHE_AGE)) {
await deleteDirectory(`${UPLOAD_CACHE_DIRECTORY}/${name}`);
}
}
};
+2
View File
@@ -0,0 +1,2 @@
export * from "./file-cache.ts";
export * from "./thumbnail-cache.svelte.ts";
+86
View File
@@ -0,0 +1,86 @@
/* eslint-disable svelte/prefer-svelte-reactivity -- Lookup tables of reactive entries */
import { LRUCache } from "lru-cache";
import { deleteDirectory, deleteFile, readFile, writeFile } from "#lib/client/storage/opfs.ts";
import { decryptChunk, encodeToBase64 } from "#lib/shared/crypto/index.ts";
export interface Thumbnail {
url?: string;
}
const getThumbnailUrl = (thumbnailBuffer: ArrayBuffer) => {
return `data:image/webp;base64,${encodeToBase64(thumbnailBuffer)}`;
};
// Thumbnails are reactive so that they can be updated in place
const loadedThumbnails = new LRUCache<number, Thumbnail>({ max: 100 });
const loadingThumbnails = new Map<number, Thumbnail>();
const fetchFromOpfs = async (fileId: number) => {
const thumbnailBuffer = await readFile(`/thumbnail/file/${fileId}`);
if (thumbnailBuffer) {
return getThumbnailUrl(thumbnailBuffer);
}
};
const fetchFromServer = async (fileId: number, dataKey: CryptoKey) => {
const res = await fetch(`/api/file/${fileId}/thumbnail/download`);
if (!res.ok) return null;
const thumbnailBuffer = await decryptChunk(await res.arrayBuffer(), dataKey);
void writeFile(`/thumbnail/file/${fileId}`, thumbnailBuffer);
return getThumbnailUrl(thumbnailBuffer);
};
export const getFileThumbnail = (file: {
id: number;
contentType: string;
dataKey?: { key: CryptoKey };
}) => {
if (!(file.contentType.startsWith("image/") || file.contentType.startsWith("video/"))) {
return undefined;
}
const thumbnail = loadedThumbnails.get(file.id) ?? loadingThumbnails.get(file.id);
if (thumbnail) return thumbnail;
const loadingThumbnail: Thumbnail = $state({});
loadingThumbnails.set(file.id, loadingThumbnail);
fetchFromOpfs(file.id)
.then((url) => url ?? (file.dataKey && fetchFromServer(file.id, file.dataKey.key)))
.then((url) => {
if (url) {
loadingThumbnail.url = url;
loadedThumbnails.set(file.id, loadingThumbnail);
}
})
.catch(console.error)
.finally(() => {
loadingThumbnails.delete(file.id);
});
return loadingThumbnail;
};
export const storeFileThumbnailCache = async (fileId: number, thumbnailBuffer: ArrayBuffer) => {
await writeFile(`/thumbnail/file/${fileId}`, thumbnailBuffer);
const url = getThumbnailUrl(thumbnailBuffer);
const thumbnail = loadedThumbnails.get(fileId) ?? loadingThumbnails.get(fileId);
if (thumbnail) {
thumbnail.url = url;
} else {
const newThumbnail: Thumbnail = $state({ url });
loadedThumbnails.set(fileId, newThumbnail);
}
};
export const deleteFileThumbnailCache = async (fileId: number) => {
loadedThumbnails.delete(fileId);
await deleteFile(`/thumbnail/file/${fileId}`);
};
export const deleteAllFileThumbnailCaches = async () => {
loadedThumbnails.clear();
await deleteDirectory(`/thumbnail/file`);
};
+29
View File
@@ -0,0 +1,29 @@
import type { ComputeMessage, ResultMessage } from "#workers/hmac.ts";
import HmacWorker from "#workers/hmac.ts?worker";
// Computes HMAC of large blobs incrementally in a worker, which WebCrypto doesn't support
export const signMessageHmac = async (message: Blob, hmacSecret: CryptoKey) => {
const stream = message.stream();
const hmacSecretRaw = new Uint8Array(await crypto.subtle.exportKey("raw", hmacSecret));
const worker = new HmacWorker();
return new Promise<Uint8Array>((resolve, reject) => {
worker.onmessage = ({ data }: MessageEvent<ResultMessage>) => {
if ("result" in data) {
resolve(data.result);
} else {
reject(data.error);
}
worker.terminate();
};
worker.onerror = ({ error }) => {
reject(error);
worker.terminate();
};
worker.postMessage({ stream, key: hmacSecretRaw } satisfies ComputeMessage, {
transfer: [stream, hmacSecretRaw.buffer],
});
});
};
@@ -1,31 +1,28 @@
import { import {
generateEncryptionKeyPair, concatenateBuffers,
generateSigningKeyPair, digestMessage,
exportRSAKey,
exportRSAKeyToBase64, exportRSAKeyToBase64,
generateEncryptionKeyPair,
generateHmacSecret,
generateMasterKey,
generateSigningKeyPair,
importEncryptionKeyPairFromBase64, importEncryptionKeyPairFromBase64,
importSigningKeyPairFromBase64, importSigningKeyPairFromBase64,
makeRSAKeyNonextractable,
wrapMasterKey,
generateMasterKey,
makeAESKeyNonextractable, makeAESKeyNonextractable,
makeRSAKeyNonextractable,
wrapHmacSecret, wrapHmacSecret,
generateHmacSecret, wrapMasterKey,
} from "$lib/modules/crypto"; } from "#lib/shared/crypto/index.ts";
import { deserializeClientKeys } from "$lib/modules/key"; import { keyring } from "./keyring.svelte.ts";
import { clientKeyStore } from "$lib/stores"; import { deserializeClientKeys } from "./serialization.ts";
export { requestLogout } from "$lib/services/auth";
export { requestDeletedFilesCleanup } from "$lib/services/file";
export {
requestClientRegistrationAndSessionUpgrade,
requestInitialMasterKeyAndHmacSecretRegistration,
} from "$lib/services/key";
// Generates new client keys and keeps them in the keyring without persisting
export const generateClientKeys = async () => { export const generateClientKeys = async () => {
const { encryptKey, decryptKey } = await generateEncryptionKeyPair(); const { encryptKey, decryptKey } = await generateEncryptionKeyPair();
const { signKey, verifyKey } = await generateSigningKeyPair(); const { signKey, verifyKey } = await generateSigningKeyPair();
clientKeyStore.set({ keyring.setClientKeys({
encryptKey, encryptKey,
decryptKey: await makeRSAKeyNonextractable(decryptKey), decryptKey: await makeRSAKeyNonextractable(decryptKey),
signKey: await makeRSAKeyNonextractable(signKey), signKey: await makeRSAKeyNonextractable(signKey),
@@ -41,6 +38,28 @@ export const generateClientKeys = async () => {
}; };
}; };
export const importClientKeys = async (clientKeysSerialized: string) => {
const clientKeys = deserializeClientKeys(clientKeysSerialized);
if (!clientKeys) return false;
const { encryptKey, decryptKey } = await importEncryptionKeyPairFromBase64(
clientKeys.encryptKeyBase64,
clientKeys.decryptKeyBase64,
);
const { signKey, verifyKey } = await importSigningKeyPairFromBase64(
clientKeys.signKeyBase64,
clientKeys.verifyKeyBase64,
);
keyring.setClientKeys({
encryptKey,
decryptKey: await makeRSAKeyNonextractable(decryptKey),
signKey: await makeRSAKeyNonextractable(signKey),
verifyKey,
});
return true;
};
export const generateInitialMasterKey = async (encryptKey: CryptoKey) => { export const generateInitialMasterKey = async (encryptKey: CryptoKey) => {
const { masterKey } = await generateMasterKey(); const { masterKey } = await generateMasterKey();
return { return {
@@ -56,24 +75,14 @@ export const generateInitialHmacSecret = async (masterKey: CryptoKey) => {
}; };
}; };
export const importClientKeys = async (clientKeysSerialized: string) => { export const generatePublicKeyFingerprint = async (encryptKey: CryptoKey, verifyKey: CryptoKey) => {
const clientKeys = deserializeClientKeys(clientKeysSerialized); const { key: encryptKeyBuffer } = await exportRSAKey(encryptKey);
if (!clientKeys) return false; const { key: verifyKeyBuffer } = await exportRSAKey(verifyKey);
const digest = await digestMessage(concatenateBuffers(encryptKeyBuffer, verifyKeyBuffer));
const { encryptKey, decryptKey } = await importEncryptionKeyPairFromBase64( return Array.from(new Uint8Array(digest))
clientKeys.encryptKeyBase64, .map((byte) => byte.toString(16).padStart(2, "0"))
clientKeys.decryptKeyBase64, .join("")
); .toUpperCase()
const { signKey, verifyKey } = await importSigningKeyPairFromBase64( .match(/.{1,4}/g)!
clientKeys.signKeyBase64, .join(" ");
clientKeys.verifyKeyBase64,
);
clientKeyStore.set({
encryptKey,
decryptKey: await makeRSAKeyNonextractable(decryptKey),
signKey: await makeRSAKeyNonextractable(signKey),
verifyKey,
});
return true;
}; };
+5
View File
@@ -0,0 +1,5 @@
export * from "./generation.ts";
export * from "./keyring.svelte.ts";
export * from "./serialization.ts";
export * from "./sync.ts";
export * from "./types.ts";
+113
View File
@@ -0,0 +1,113 @@
import {
getClientKey,
getHmacSecrets,
getMasterKeys,
storeClientKey,
storeHmacSecrets,
storeMasterKeys,
} from "#lib/client/storage/indexed-db/index.ts";
import type { ClientKeys, HmacSecret, MasterKey } from "./types.ts";
const findLatestActive = <T extends { version: number; state: string }>(
entries: ReadonlyMap<number, T>,
) => {
return entries
.values()
.filter(({ state }) => state === "active")
.reduce<T | undefined>(
(latest, entry) => ((latest?.version ?? 0) > entry.version ? latest : entry),
undefined,
);
};
// Holds the keys of this device and the user. Keys are persisted in IndexedDB.
class Keyring {
clientKeys = $state.raw<ClientKeys | null>(null);
masterKeys = $state.raw<ReadonlyMap<number, MasterKey>>(new Map());
hmacSecrets = $state.raw<ReadonlyMap<number, HmacSecret>>(new Map());
async load() {
const [encryptKey, decryptKey, signKey, verifyKey, masterKeys, hmacSecrets] = await Promise.all(
[
getClientKey("encrypt"),
getClientKey("decrypt"),
getClientKey("sign"),
getClientKey("verify"),
getMasterKeys(),
getHmacSecrets(),
],
);
if (encryptKey && decryptKey && signKey && verifyKey) {
this.clientKeys = { encryptKey, decryptKey, signKey, verifyKey };
}
this.masterKeys = new Map(masterKeys.map((masterKey) => [masterKey.version, masterKey]));
this.hmacSecrets = new Map(hmacSecrets.map((hmacSecret) => [hmacSecret.version, hmacSecret]));
}
// Client keys are kept in memory only until they are registered and persisted
setClientKeys(clientKeys: ClientKeys) {
this.clientKeys = clientKeys;
}
async persistClientKeys() {
const clientKeys = this.requireClientKeys();
await Promise.all([
storeClientKey(clientKeys.encryptKey, "encrypt"),
storeClientKey(clientKeys.decryptKey, "decrypt"),
storeClientKey(clientKeys.signKey, "sign"),
storeClientKey(clientKeys.verifyKey, "verify"),
]);
}
async setMasterKeys(masterKeys: MasterKey[]) {
await storeMasterKeys(masterKeys);
this.masterKeys = new Map(masterKeys.map((masterKey) => [masterKey.version, masterKey]));
}
async setHmacSecrets(hmacSecrets: HmacSecret[]) {
await storeHmacSecrets(hmacSecrets);
this.hmacSecrets = new Map(hmacSecrets.map((hmacSecret) => [hmacSecret.version, hmacSecret]));
}
get hasMasterKeys() {
return this.masterKeys.size > 0;
}
get hasHmacSecrets() {
return this.hmacSecrets.size > 0;
}
requireClientKeys() {
if (!this.clientKeys) {
throw new Error("Client keys not found");
}
return this.clientKeys;
}
getMasterKey(version: number) {
const masterKey = this.masterKeys.get(version);
if (!masterKey) {
throw new Error(`Master key (version ${version}) not found`);
}
return masterKey;
}
// The master key used to encrypt new data
get activeMasterKey() {
const masterKey = findLatestActive(this.masterKeys);
if (!masterKey) {
throw new Error("Active master key not found");
}
return masterKey;
}
get activeHmacSecret() {
const hmacSecret = findLatestActive(this.hmacSecrets);
if (!hmacSecret) {
throw new Error("Active HMAC secret not found");
}
return hmacSecret;
}
}
export const keyring = new Keyring();
@@ -1,8 +1,6 @@
import { z } from "zod"; import { z } from "zod";
import { storeClientKey } from "$lib/indexedDB";
import type { ClientKeys } from "$lib/stores";
const serializedClientKeysSchema = z.intersection( const SerializedClientKeysSchema = z.intersection(
z.object({ z.object({
generator: z.literal("ArkVault"), generator: z.literal("ArkVault"),
exportedAt: z.iso.datetime(), exportedAt: z.iso.datetime(),
@@ -16,7 +14,7 @@ const serializedClientKeysSchema = z.intersection(
}), }),
); );
type SerializedClientKeys = z.infer<typeof serializedClientKeysSchema>; type SerializedClientKeys = z.infer<typeof SerializedClientKeysSchema>;
type DeserializedClientKeys = { type DeserializedClientKeys = {
encryptKeyBase64: string; encryptKeyBase64: string;
@@ -43,7 +41,7 @@ export const serializeClientKeys = ({
}; };
export const deserializeClientKeys = (serialized: string) => { export const deserializeClientKeys = (serialized: string) => {
const zodRes = serializedClientKeysSchema.safeParse(JSON.parse(serialized)); const zodRes = SerializedClientKeysSchema.safeParse(JSON.parse(serialized));
if (zodRes.success) { if (zodRes.success) {
return { return {
encryptKeyBase64: zodRes.data.encryptKey, encryptKeyBase64: zodRes.data.encryptKey,
@@ -54,12 +52,3 @@ export const deserializeClientKeys = (serialized: string) => {
} }
return undefined; return undefined;
}; };
export const storeClientKeys = async (clientKeys: ClientKeys) => {
await Promise.all([
storeClientKey(clientKeys.encryptKey, "encrypt"),
storeClientKey(clientKeys.decryptKey, "decrypt"),
storeClientKey(clientKeys.signKey, "sign"),
storeClientKey(clientKeys.verifyKey, "verify"),
]);
};
+182
View File
@@ -0,0 +1,182 @@
import {
decryptChallenge,
encodeToBase64,
exportRSAKeyToBase64,
signMasterKeyWrapped,
signMessageRSA,
unwrapHmacSecret,
unwrapMasterKey,
verifyMasterKeyWrapped,
} from "#lib/shared/crypto/index.ts";
import { isTRPCClientError, trpc } from "#trpc/client.ts";
import { keyring } from "./keyring.svelte.ts";
import type { ClientKeys } from "./types.ts";
// Synchronizes keys with the server
const exportPublicKeys = async ({ encryptKey, verifyKey }: ClientKeys) => ({
encPubKey: await exportRSAKeyToBase64(encryptKey),
sigPubKey: await exportRSAKeyToBase64(verifyKey),
});
const answerChallenge = async (challenge: string, { decryptKey, signKey }: ClientKeys) => {
const answer = await decryptChallenge(challenge, decryptKey);
return encodeToBase64(await signMessageRSA(answer, signKey));
};
export const requestSessionUpgrade = async (force = false) => {
const clientKeys = keyring.requireClientKeys();
let id, challenge;
try {
({ id, challenge } = await trpc.auth.upgrade.mutate(await exportPublicKeys(clientKeys)));
} catch (e) {
if (isTRPCClientError(e) && e.data?.code === "FORBIDDEN") {
return [false, "Unregistered client"] as const;
}
return [false] as const;
}
try {
await trpc.auth.verifyUpgrade.mutate({
id,
answerSig: await answerChallenge(challenge, clientKeys),
force,
});
} catch (e) {
if (isTRPCClientError(e) && e.data?.code === "CONFLICT") {
return [false, "Already logged in"] as const;
}
return [false] as const;
}
return [true] as const;
};
export const requestClientRegistration = async () => {
const clientKeys = keyring.requireClientKeys();
try {
const { id, challenge } = await trpc.client.register.mutate(await exportPublicKeys(clientKeys));
await trpc.client.verify.mutate({
id,
answerSig: await answerChallenge(challenge, clientKeys),
});
return true;
} catch (e) {
console.error(e);
return false;
}
};
export const requestClientRegistrationAndSessionUpgrade = async (force: boolean) => {
const [res, error] = await requestSessionUpgrade(force);
if (error === undefined) return [res] as const;
if (error === "Unregistered client" && !(await requestClientRegistration())) {
return [false] as const;
} else if (error === "Already logged in") {
return [false, force ? undefined : error] as const;
}
return [(await requestSessionUpgrade())[0]] as const;
};
export const registerInitialMasterKeyAndHmacSecret = async (
masterKeyWrapped: string,
hmacSecretWrapped: string,
) => {
const { signKey } = keyring.requireClientKeys();
try {
await trpc.mek.registerInitial.mutate({
mek: masterKeyWrapped,
mekSig: await signMasterKeyWrapped(masterKeyWrapped, 1, signKey),
});
} catch (e) {
if (isTRPCClientError(e) && (e.data?.code === "FORBIDDEN" || e.data?.code === "CONFLICT")) {
return true; // Already registered by another client
}
console.error(e);
return false;
}
try {
await trpc.hsk.registerInitial.mutate({
mekVersion: 1,
hsk: hmacSecretWrapped,
});
return true;
} catch (e) {
console.error(e);
return false;
}
};
export const downloadHmacSecrets = async () => {
let hmacSecretsWrapped;
try {
hmacSecretsWrapped = await trpc.hsk.list.query();
} catch (e) {
console.error(e);
return false;
}
const hmacSecrets = await Promise.all(
hmacSecretsWrapped.map(async ({ version, state, mekVersion, hsk: hmacSecretWrapped }) => {
const { hmacSecret } = await unwrapHmacSecret(
hmacSecretWrapped,
keyring.getMasterKey(mekVersion).key,
);
return { version, state, secret: hmacSecret };
}),
);
await keyring.setHmacSecrets(hmacSecrets);
return true;
};
// Downloads the master keys (and HMAC secrets) once this client has been approved
export const downloadMasterKeys = async () => {
const { decryptKey, verifyKey } = keyring.requireClientKeys();
let masterKeysWrapped;
try {
masterKeysWrapped = await trpc.mek.list.query();
} catch (e) {
console.error(e);
return false;
}
const masterKeys = await Promise.all(
masterKeysWrapped.map(
async ({ version, state, mek: masterKeyWrapped, mekSig: masterKeyWrappedSig }) => {
const { masterKey } = await unwrapMasterKey(masterKeyWrapped, decryptKey);
return {
version,
state,
key: masterKey,
isValid: await verifyMasterKeyWrapped(
masterKeyWrapped,
version,
masterKeyWrappedSig,
verifyKey,
),
};
},
),
);
if (masterKeys.length === 0 || !masterKeys.every(({ isValid }) => isValid)) return false;
await keyring.setMasterKeys(
masterKeys.map(({ version, state, key }) => ({ version, state, key })),
);
await downloadHmacSecrets();
return true;
};
export const ensureHmacSecrets = async () => {
if (!keyring.hasHmacSecrets && !(await downloadHmacSecrets())) {
throw new Error("Failed to download HMAC secrets");
}
};
+18
View File
@@ -0,0 +1,18 @@
export interface ClientKeys {
encryptKey: CryptoKey;
decryptKey: CryptoKey;
signKey: CryptoKey;
verifyKey: CryptoKey;
}
export interface MasterKey {
version: number;
state: "active" | "retired";
key: CryptoKey;
}
export interface HmacSecret {
version: number;
state: "active";
secret: CryptoKey;
}
+52
View File
@@ -0,0 +1,52 @@
import ExifReader from "exifreader";
const isInteger = (value: number | undefined): value is number => Number.isInteger(value);
export const extractExifDateTime = (fileBuffer: ArrayBuffer) => {
const exif = ExifReader.load(fileBuffer);
const dateTimeOriginal = exif["DateTimeOriginal"]?.description;
const offsetTimeOriginal = exif["OffsetTimeOriginal"]?.description;
if (!dateTimeOriginal) return undefined;
const [date, time] = dateTimeOriginal.split(" ");
if (!date || !time) return undefined;
const [year, month, day] = date.split(":").map(Number);
const [hour, minute, second] = time.split(":").map(Number);
// Hours, minutes, and seconds may be 0, so they are checked differently
if (!year || !month || !day || !isInteger(hour) || !isInteger(minute) || !isInteger(second)) {
return undefined;
}
if (!offsetTimeOriginal) {
// No timezone information.. Assume local timezone
return new Date(year, month - 1, day, hour, minute, second);
}
const offsetSign = offsetTimeOriginal[0] === "+" ? 1 : -1;
const [offsetHour, offsetMinute] = offsetTimeOriginal.slice(1).split(":").map(Number);
const utcDate = Date.UTC(year, month - 1, day, hour, minute, second);
const offsetMs = offsetSign * ((offsetHour ?? 0) * 60 + (offsetMinute ?? 0)) * 60 * 1000;
return new Date(utcDate - offsetMs);
};
const EXIF_SEARCH_SIZE = 1024 * 1024; // 1 MiB
const MAX_FULL_READ_SIZE = 64 * 1024 * 1024; // 64 MiB
const tryExtractExifDateTime = (buffer: ArrayBuffer) => {
try {
return extractExifDateTime(buffer);
} catch {
return undefined;
}
};
// EXIF is usually at the beginning of the file, so the whole file is read only if not found
export const readExifDateTime = async (file: Blob) => {
const dateTime = tryExtractExifDateTime(await file.slice(0, EXIF_SEARCH_SIZE).arrayBuffer());
if (dateTime || file.size <= EXIF_SEARCH_SIZE || file.size > MAX_FULL_READ_SIZE) {
return dateTime;
}
return tryExtractExifDateTime(await file.arrayBuffer());
};
+5
View File
@@ -0,0 +1,5 @@
// Chromium can't decode HEIC images, so they are converted to be displayed
export const convertHeic = async (blob: Blob, toType: "image/png" | "image/jpeg") => {
const { default: heic2any } = await import("heic2any");
return (await heic2any({ blob, toType })) as Blob;
};
@@ -1,4 +1,4 @@
import { encodeToBase64 } from "$lib/modules/crypto"; import { convertHeic } from "./heic.ts";
const scaleSize = (width: number, height: number, targetSize: number) => { const scaleSize = (width: number, height: number, targetSize: number) => {
if (width <= targetSize || height <= targetSize) { if (width <= targetSize || height <= targetSize) {
@@ -52,7 +52,6 @@ const generateImageThumbnail = (imageUrl: string) => {
.catch(reject); .catch(reject);
}; };
image.onerror = reject; image.onerror = reject;
image.src = imageUrl; image.src = imageUrl;
}); });
}; };
@@ -63,65 +62,88 @@ export const captureVideoThumbnail = (video: HTMLVideoElement) => {
}); });
}; };
const VIDEO_THUMBNAIL_TIMEOUT_MS = 30_000;
const generateVideoThumbnail = (videoUrl: string, time = 0) => { const generateVideoThumbnail = (videoUrl: string, time = 0) => {
return new Promise<Blob>((resolve, reject) => { return new Promise<Blob>((resolve, reject) => {
const video = document.createElement("video"); const video = document.createElement("video");
let isCaptured = false;
const cleanup = () => {
clearTimeout(timeout);
video.onloadedmetadata = video.onloadeddata = video.onseeked = video.onerror = null;
video.removeAttribute("src");
video.load(); // Stops loading the video
};
const fail = (reason: unknown) => {
cleanup();
reject(reason);
};
const captureFrame = () => {
if (isCaptured || video.readyState < HTMLMediaElement.HAVE_CURRENT_DATA) return;
isCaptured = true;
captureVideoThumbnail(video).then(resolve, reject).finally(cleanup);
};
// Streamed videos may stall without any error
const timeout = setTimeout(
() => fail(new Error("Timed out while loading the video")),
VIDEO_THUMBNAIL_TIMEOUT_MS,
);
video.onloadedmetadata = () => { video.onloadedmetadata = () => {
if (video.videoWidth === 0 || video.videoHeight === 0) { if (video.videoWidth === 0 || video.videoHeight === 0) {
return reject(); return fail(new Error("Invalid video size"));
} }
const callbackId = video.requestVideoFrameCallback(() => { // Frame callbacks aren't guaranteed for videos not in the document, so also wait for seeking
captureVideoThumbnail(video).then(resolve).catch(reject); video.requestVideoFrameCallback(captureFrame);
video.cancelVideoFrameCallback(callbackId); video.onseeked = captureFrame;
}); if (time === 0) {
video.onloadeddata = captureFrame;
}
video.currentTime = Math.min(time, video.duration); video.currentTime = Math.min(time, video.duration);
}; };
video.onerror = reject; video.onerror = () => fail(video.error);
video.muted = true; video.muted = true;
video.playsInline = true; video.playsInline = true;
video.preload = "auto";
video.src = videoUrl; video.src = videoUrl;
}); });
}; };
export const generateThumbnail = async (fileBuffer: ArrayBuffer, fileType: string) => { // Generates a thumbnail of the image or video at the URL. Returns `null` if not supported.
let url; export const generateThumbnailFromUrl = async (url: string, contentType: string) => {
try { try {
if (fileType.startsWith("image/")) { if (contentType.startsWith("image/")) {
const fileBlob = new Blob([fileBuffer], { type: fileType });
url = URL.createObjectURL(fileBlob);
try { try {
return await generateImageThumbnail(url); return await generateImageThumbnail(url);
} catch { } catch {
URL.revokeObjectURL(url); if (contentType !== "image/heic") return null;
url = undefined;
if (fileType === "image/heic") { const blob = await (await fetch(url)).blob();
const { default: heic2any } = await import("heic2any"); const png = URL.createObjectURL(await convertHeic(blob, "image/png"));
url = URL.createObjectURL( try {
(await heic2any({ blob: fileBlob, toType: "image/png" })) as Blob, return await generateImageThumbnail(png);
); } finally {
return await generateImageThumbnail(url); URL.revokeObjectURL(png);
} else {
return null;
} }
} }
} else if (fileType.startsWith("video/")) { } else if (contentType.startsWith("video/")) {
url = URL.createObjectURL(new Blob([fileBuffer], { type: fileType }));
return await generateVideoThumbnail(url); return await generateVideoThumbnail(url);
} }
return null; return null;
} catch { } catch {
return null; return null;
} finally {
if (url) {
URL.revokeObjectURL(url);
}
} }
}; };
export const getThumbnailUrl = (thumbnailBuffer: ArrayBuffer) => { export const generateThumbnail = async (blob: Blob) => {
return `data:image/webp;base64,${encodeToBase64(thumbnailBuffer)}`; const url = URL.createObjectURL(blob);
try {
return await generateThumbnailFromUrl(url, blob.type);
} finally {
URL.revokeObjectURL(url);
}
}; };
+80
View File
@@ -0,0 +1,80 @@
import { LRUCache } from "lru-cache";
import { handleDownloadProgress } from "#lib/client/transfer/download.svelte.ts";
import {
DECRYPTED_FILE_URL_PREFIX,
MAX_PREPARED_FILES,
type FileMetadata,
type ServiceWorkerMessage,
type ServiceWorkerResponse,
} from "#lib/shared/service-worker.ts";
const PREPARE_TIMEOUT_MS = 5000;
// The service worker may be terminated at any time, losing its in-memory metadata.
// Keep a copy here so that it can be re-requested on demand.
const preparedFiles = new LRUCache<number, FileMetadata>({ max: MAX_PREPARED_FILES });
let isListening = false;
export const listenForServiceWorkerMessages = () => {
if (isListening) return;
isListening = true;
navigator.serviceWorker.addEventListener(
"message",
(event: MessageEvent<ServiceWorkerResponse>) => {
switch (event.data.type) {
case "decryption-metadata-request":
event.ports[0]?.postMessage(preparedFiles.get(event.data.fileId) ?? null);
break;
case "download-progress":
handleDownloadProgress(event.data);
break;
}
},
);
};
const getServiceWorker = async () => {
const registration = await navigator.serviceWorker.ready;
const sw = registration.active;
if (!sw) {
throw new Error("Service worker not activated");
}
return sw;
};
export const prepareFileDecryption = async (id: number, metadata: FileMetadata) => {
preparedFiles.set(id, metadata);
listenForServiceWorkerMessages();
const sw = await getServiceWorker();
return new Promise<void>((resolve, reject) => {
const timeout = setTimeout(
() => reject(new Error("Service worker timeout")),
PREPARE_TIMEOUT_MS,
);
const handler = (event: MessageEvent<ServiceWorkerResponse>) => {
if (event.data.type === "decryption-ready" && event.data.fileId === id) {
clearTimeout(timeout);
navigator.serviceWorker.removeEventListener("message", handler);
resolve();
}
};
navigator.serviceWorker.addEventListener("message", handler);
sw.postMessage({
type: "decryption-prepare",
fileId: id,
...metadata,
} satisfies ServiceWorkerMessage);
});
};
export const getDecryptedFileUrl = (id: number) => `${DECRYPTED_FILE_URL_PREFIX}${id}`;
// Lets the service worker download, decrypt, and save the file
export const saveDecryptedFile = (id: number, filename: string) => {
const link = document.createElement("a");
link.href = `${getDecryptedFileUrl(id)}?download=${encodeURIComponent(filename)}`;
link.click();
};
@@ -0,0 +1,46 @@
import { Dexie, type EntityTable } from "dexie";
export interface FileCacheIndex {
fileId: number;
cachedAt: Date;
lastRetrievedAt: Date;
// Files are cached in chunks, possibly partially (e.g. videos which were partially played)
size: number;
totalSize: number;
chunkCount: number;
isComplete: boolean;
}
const cacheIndex = new Dexie("cacheIndex") as Dexie & {
fileCache: EntityTable<FileCacheIndex, "fileId">;
};
// Files were cached as single files before version 2
cacheIndex
.version(2)
.stores({
fileCache: "fileId",
})
.upgrade(async (trx) => {
await trx.table("fileCache").clear();
});
export const getFileCacheIndex = async () => {
return await cacheIndex.fileCache.toArray();
};
export const getFileCacheIndexOf = async (fileId: number) => {
return await cacheIndex.fileCache.get(fileId);
};
export const storeFileCacheIndex = async (fileCacheIndex: FileCacheIndex) => {
await cacheIndex.fileCache.put(fileCacheIndex);
};
export const updateFileCacheRetrievedAt = async (fileId: number, lastRetrievedAt: Date) => {
await cacheIndex.fileCache.update(fileId, { lastRetrievedAt });
};
export const deleteFileCacheIndex = async (fileId: number) => {
await cacheIndex.fileCache.delete(fileId);
};
@@ -4,6 +4,7 @@ interface DirectoryInfo {
id: number; id: number;
parentId: DirectoryId; parentId: DirectoryId;
name: string; name: string;
isFavorite: boolean;
} }
interface FileInfo { interface FileInfo {
@@ -14,6 +15,7 @@ interface FileInfo {
createdAt?: Date; createdAt?: Date;
lastModifiedAt: Date; lastModifiedAt: Date;
categoryIds?: number[]; categoryIds?: number[];
isFavorite: boolean;
} }
interface CategoryInfo { interface CategoryInfo {
@@ -46,6 +48,23 @@ filesystem
}); });
}); });
filesystem.version(4).upgrade(async (trx) => {
await Promise.all([
trx
.table("directory")
.toCollection()
.modify((directory) => {
directory.isFavorite = false;
}),
trx
.table("file")
.toCollection()
.modify((file) => {
file.isFavorite = false;
}),
]);
});
export const getDirectoryInfos = async (parentId: DirectoryId) => { export const getDirectoryInfos = async (parentId: DirectoryId) => {
return await filesystem.directory.where({ parentId }).toArray(); return await filesystem.directory.where({ parentId }).toArray();
}; };
@@ -55,7 +74,11 @@ export const getDirectoryInfo = async (id: number) => {
}; };
export const storeDirectoryInfo = async (directoryInfo: DirectoryInfo) => { export const storeDirectoryInfo = async (directoryInfo: DirectoryInfo) => {
await filesystem.directory.upsert(directoryInfo.id, { ...directoryInfo }); await filesystem.directory.upsert(directoryInfo.id, {
parentId: directoryInfo.parentId,
name: directoryInfo.name,
isFavorite: directoryInfo.isFavorite,
});
}; };
export const deleteDirectoryInfo = async (id: number) => { export const deleteDirectoryInfo = async (id: number) => {
@@ -89,7 +112,15 @@ export const bulkGetFileInfos = async (ids: number[]) => {
}; };
export const storeFileInfo = async (fileInfo: FileInfo) => { export const storeFileInfo = async (fileInfo: FileInfo) => {
await filesystem.file.upsert(fileInfo.id, { ...fileInfo }); await filesystem.file.upsert(fileInfo.id, {
parentId: fileInfo.parentId,
name: fileInfo.name,
contentType: fileInfo.contentType,
createdAt: fileInfo.createdAt,
lastModifiedAt: fileInfo.lastModifiedAt,
categoryIds: fileInfo.categoryIds,
isFavorite: fileInfo.isFavorite,
});
}; };
export const deleteFileInfo = async (id: number) => { export const deleteFileInfo = async (id: number) => {
@@ -116,7 +147,12 @@ export const getCategoryInfo = async (id: number) => {
}; };
export const storeCategoryInfo = async (categoryInfo: CategoryInfo) => { export const storeCategoryInfo = async (categoryInfo: CategoryInfo) => {
await filesystem.category.upsert(categoryInfo.id, { ...categoryInfo }); await filesystem.category.upsert(categoryInfo.id, {
parentId: categoryInfo.parentId,
name: categoryInfo.name,
files: categoryInfo.files,
isFileRecursive: categoryInfo.isFileRecursive,
});
}; };
export const updateCategoryInfo = async (id: number, changes: { isFileRecursive?: boolean }) => { export const updateCategoryInfo = async (id: number, changes: { isFileRecursive?: boolean }) => {
@@ -0,0 +1,3 @@
export * from "./cache-index.ts";
export * from "./filesystem.ts";
export * from "./key-store.ts";
@@ -70,12 +70,12 @@ export const storeMasterKeys = async (keys: MasterKey[]) => {
}; };
export const getHmacSecrets = async () => { export const getHmacSecrets = async () => {
return await keyStore.hmacSecret.toArray(); return (await keyStore.hmacSecret.toArray()).filter(({ secret }) => secret.extractable);
}; };
export const storeHmacSecrets = async (secrets: HmacSecret[]) => { export const storeHmacSecrets = async (secrets: HmacSecret[]) => {
if (secrets.some(({ secret }) => secret.extractable)) { if (secrets.some(({ secret }) => !secret.extractable)) {
throw new Error("Hmac secrets must be nonextractable"); throw new Error("Hmac secrets must be extractable");
} }
await keyStore.hmacSecret.bulkPut(secrets); await keyStore.hmacSecret.bulkPut(secrets);
}; };
@@ -48,6 +48,12 @@ export const writeFile = async (path: string, data: ArrayBuffer) => {
} }
}; };
// Changes are applied atomically when the stream is closed, and discarded if it is aborted
export const createWritable = async (path: string) => {
const { fileHandle } = await getFileHandle(path);
return await fileHandle!.createWritable();
};
export const deleteFile = async (path: string) => { export const deleteFile = async (path: string) => {
const { parentHandle, filename } = await getFileHandle(path, false); const { parentHandle, filename } = await getFileHandle(path, false);
if (!parentHandle) return; if (!parentHandle) return;
@@ -88,3 +94,51 @@ export const deleteDirectory = async (path: string) => {
await parentHandle.removeEntry(directoryHandle.name, { recursive: true }); await parentHandle.removeEntry(directoryHandle.name, { recursive: true });
}; };
const listEntryNames = async (path: string, kind: FileSystemHandleKind) => {
const { directoryHandle } = await getDirectoryHandle(path);
if (!directoryHandle) return [];
const names: string[] = [];
for await (const handle of directoryHandle.values()) {
if (handle.kind === kind) {
names.push(handle.name);
}
}
return names;
};
export const listFileNames = (path: string) => listEntryNames(path, "file");
export const listDirectoryNames = (path: string) => listEntryNames(path, "directory");
// Deletes the files (not directories) directly under the directory
export const deleteFilesInDirectory = async (path: string) => {
const { directoryHandle } = await getDirectoryHandle(path);
if (!directoryHandle) return;
for (const name of await listFileNames(path)) {
await directoryHandle.removeEntry(name);
}
};
// Renames the file atomically
export const moveFile = async (path: string, newPath: string) => {
const { fileHandle } = await getFileHandle(path, false);
if (!fileHandle) {
throw new Error("File not found");
}
let directoryHandle = await navigator.storage.getDirectory();
const parts = newPath.split("/").filter((part) => part);
for (const part of parts.slice(0, -1)) {
directoryHandle = await directoryHandle.getDirectoryHandle(part, { create: true });
}
// Supported by Chromium for the origin private file system
await (
fileHandle as FileSystemFileHandle & {
move(destination: FileSystemDirectoryHandle, name: string): Promise<void>;
}
).move(directoryHandle, parts.at(-1)!);
};
+130
View File
@@ -0,0 +1,130 @@
import { sha256 } from "@noble/hashes/sha2.js";
import axios from "axios";
import { encodeToBase64, encryptChunk } from "#lib/shared/crypto/index.ts";
import { CHUNK_SIZE, getChunkCount, getEncryptedSize } from "#lib/shared/format/index.ts";
import { BoundedQueue } from "#lib/utils/concurrency/index.ts";
import { createSpeedMeter } from "#lib/utils/speed-meter.ts";
interface UploadStats {
progress: number;
rate: number;
}
interface EncryptedChunk {
index: number;
data: ArrayBuffer;
}
const uploadEncryptedChunk = async (
uploadId: string,
chunkIndex: number,
encrypted: ArrayBuffer,
onChunkProgress: (chunkIndex: number, loaded: number) => void,
) => {
await axios.post(`/api/upload/${uploadId}/chunks/${chunkIndex + 1}`, encrypted, {
headers: { "Content-Type": "application/octet-stream" },
onUploadProgress(e) {
onChunkProgress(chunkIndex, e.loaded ?? 0);
},
});
onChunkProgress(chunkIndex, encrypted.byteLength);
};
export const uploadBlob = async (
uploadId: string,
blob: Blob,
dataKey: CryptoKey,
options?: {
concurrency?: number;
onProgress?: (s: UploadStats) => void;
// Receives each chunk of the plaintext before it is encrypted
onPlaintextChunk?: (chunkIndex: number, data: ArrayBuffer) => Promise<void>;
},
) => {
const onProgress = options?.onProgress;
const networkConcurrency = options?.concurrency ?? 4;
const maxQueueSize = 8;
const totalChunks = getChunkCount(blob.size);
const totalBytes = getEncryptedSize(blob.size);
const uploadedByChunk = new Array<number>(totalChunks).fill(0);
const speedMeter = createSpeedMeter(3000, 200);
const hash = sha256.create();
const emit = () => {
if (!onProgress) return;
const uploadedBytes = uploadedByChunk.reduce((a, b) => a + b, 0);
const rate = speedMeter(uploadedBytes);
const progress = Math.min(1, uploadedBytes / totalBytes);
onProgress({ progress, rate });
};
const onChunkProgress = (idx: number, loaded: number) => {
uploadedByChunk[idx] = loaded;
emit();
};
const queue = new BoundedQueue<EncryptedChunk>(maxQueueSize);
// Producer: encrypt chunks and push to queue
const encryptionProducer = async () => {
try {
for (let i = 0; i < totalChunks; i++) {
const chunk = await blob.slice(i * CHUNK_SIZE, (i + 1) * CHUNK_SIZE).arrayBuffer();
await options?.onPlaintextChunk?.(i, chunk);
const data = await encryptChunk(chunk, dataKey);
hash.update(new Uint8Array(data));
await queue.push({ index: i, data });
}
} finally {
queue.close();
}
};
// Consumer: upload chunks from queue with concurrency limit
const uploadConsumer = async () => {
const activeTasks = new Set<Promise<void>>();
let failure: { reason: unknown } | undefined;
try {
while (!failure) {
const item = await queue.pop();
if (item === null) break;
const task: Promise<void> = uploadEncryptedChunk(
uploadId,
item.index,
item.data,
onChunkProgress,
)
.catch((reason) => {
failure ??= { reason };
})
.finally(() => activeTasks.delete(task));
activeTasks.add(task);
if (activeTasks.size >= networkConcurrency) {
await Promise.race(activeTasks);
}
}
await Promise.all(activeTasks);
if (failure) throw failure.reason;
} finally {
queue.close(); // Stops the producer if the upload failed
}
};
// Run producer and consumer concurrently
const [encryption, upload] = await Promise.allSettled([encryptionProducer(), uploadConsumer()]);
// The upload error takes precedence, as the producer fails due to the closed queue then
if (upload.status === "rejected") throw upload.reason;
if (encryption.status === "rejected") throw encryption.reason;
onProgress?.({ progress: 1, rate: speedMeter() });
return { encContentHash: encodeToBase64(hash.digest()) };
};
@@ -0,0 +1,55 @@
import type { DownloadProgressMessage } from "#lib/shared/service-worker.ts";
import { createSpeedMeter } from "#lib/utils/speed-meter.ts";
// Files are downloaded and decrypted by the service worker, which reports the progress
export interface FileDownloadState {
id: number;
status: "downloading" | "completed" | "error";
progress?: number;
rate?: number;
}
let downloadingFiles: FileDownloadState[] = $state([]);
const speedMeters = new WeakMap<FileDownloadState, ReturnType<typeof createSpeedMeter>>();
export const isFileDownloading = (status: FileDownloadState["status"]) => {
return status === "downloading";
};
export const getFileDownloadState = (fileId: number) => {
return downloadingFiles.find((file) => file.id === fileId && isFileDownloading(file.status));
};
export const getDownloadingFiles = () => {
return downloadingFiles.filter((file) => isFileDownloading(file.status));
};
export const clearDownloadedFiles = () => {
downloadingFiles = downloadingFiles.filter((file) => isFileDownloading(file.status));
};
export const handleDownloadProgress = ({
fileId,
status,
loaded,
total,
}: DownloadProgressMessage) => {
let state = getFileDownloadState(fileId);
if (!state) {
if (status !== "downloading") return;
downloadingFiles.push({ id: fileId, status });
state = downloadingFiles.at(-1)!;
speedMeters.set(state, createSpeedMeter());
}
if (status === "canceled") {
downloadingFiles = downloadingFiles.filter((file) => file !== state);
return;
}
state.status = status;
state.progress = total > 0 ? loaded / total : 1;
state.rate = speedMeters.get(state)?.(loaded);
};
+2
View File
@@ -0,0 +1,2 @@
export * from "./download.svelte.ts";
export * from "./upload.svelte.ts";
+259
View File
@@ -0,0 +1,259 @@
import { limitFunction } from "p-limit";
import { createUploadFileCache } from "#lib/client/cache/index.ts";
import { signMessageHmac } from "#lib/client/crypto/hmac.ts";
import {
ensureHmacSecrets,
keyring,
type HmacSecret,
type MasterKey,
} from "#lib/client/keys/index.ts";
import { readExifDateTime } from "#lib/client/media/exif.ts";
import { generateThumbnail } from "#lib/client/media/thumbnail.ts";
import { uploadBlob } from "#lib/client/transfer/chunked-upload.ts";
import {
encodeToBase64,
generateDataKey,
wrapDataKey,
encryptString,
} from "#lib/shared/crypto/index.ts";
import { getChunkCount } from "#lib/shared/format/index.ts";
import { Scheduler } from "#lib/utils/index.ts";
import { trpc } from "#trpc/client.ts";
const LIVE_STATUSES = [
"queued",
"encryption-pending",
"encrypting",
"upload-pending",
"uploading",
] as const;
type LiveStatus = (typeof LIVE_STATUSES)[number];
export interface FileUploadState {
id: string;
name: string;
parentId: DirectoryId;
status: LiveStatus | "uploaded" | "canceled" | "error";
progress?: number;
rate?: number;
estimated?: number;
}
export type LiveFileUploadState = FileUploadState & { status: LiveStatus };
const scheduler = new Scheduler<{ fileId: number; thumbnailBuffer?: ArrayBuffer } | undefined>();
let uploadingFiles: FileUploadState[] = $state([]);
const isFileUploading = (file: FileUploadState): file is LiveFileUploadState => {
return (LIVE_STATUSES as readonly string[]).includes(file.status);
};
export const getUploadingFiles = (parentId?: DirectoryId) => {
return uploadingFiles
.filter(isFileUploading)
.filter((file) => parentId === undefined || file.parentId === parentId);
};
export const clearUploadedFiles = () => {
uploadingFiles = uploadingFiles.filter(isFileUploading);
};
const requestDuplicateFileScan = limitFunction(
async (
state: FileUploadState,
file: File,
hmacSecret: HmacSecret,
onDuplicate: () => Promise<boolean>,
) => {
state.status = "encryption-pending";
const fileSigned = encodeToBase64(await signMessageHmac(file, hmacSecret.secret));
const files = await trpc.file.listByHash.query({
hskVersion: hmacSecret.version,
contentHmac: fileSigned,
});
if (files.length === 0 || (await onDuplicate())) {
return { fileSigned };
} else {
return {};
}
},
{ concurrency: 1 },
);
const getFileType = (file: File) => {
if (file.type) return file.type;
if (file.name.endsWith(".heic")) return "image/heic";
throw new Error("Unknown file type");
};
interface FileMetadata {
parentId: "root" | number;
name: string;
createdAt?: Date;
lastModifiedAt: Date;
}
const requestFileMetadataEncryption = limitFunction(
async (
state: FileUploadState,
file: Blob,
fileMetadata: FileMetadata,
masterKey: MasterKey,
hmacSecret: HmacSecret,
) => {
state.status = "encrypting";
const { dataKey, dataKeyVersion } = await generateDataKey();
const dataKeyWrapped = await wrapDataKey(dataKey, masterKey.key);
const [nameEncrypted, createdAtEncrypted, lastModifiedAtEncrypted, thumbnailBuffer] =
await Promise.all([
encryptString(fileMetadata.name, dataKey),
fileMetadata.createdAt &&
encryptString(fileMetadata.createdAt.getTime().toString(), dataKey),
encryptString(fileMetadata.lastModifiedAt.getTime().toString(), dataKey),
generateThumbnail(file).then((blob) => blob?.arrayBuffer()),
]);
const { uploadId } = await trpc.upload.startFileUpload.mutate({
chunks: getChunkCount(file.size),
parent: fileMetadata.parentId,
mekVersion: masterKey.version,
dek: dataKeyWrapped,
dekVersion: dataKeyVersion,
hskVersion: hmacSecret.version,
contentType: file.type,
name: nameEncrypted.ciphertext,
nameIv: nameEncrypted.iv,
createdAt: createdAtEncrypted?.ciphertext,
createdAtIv: createdAtEncrypted?.iv,
lastModifiedAt: lastModifiedAtEncrypted.ciphertext,
lastModifiedAtIv: lastModifiedAtEncrypted.iv,
});
state.status = "upload-pending";
return { uploadId, thumbnailBuffer, dataKey, dataKeyVersion };
},
{ concurrency: 4 },
);
const requestFileUpload = limitFunction(
async (
state: FileUploadState,
uploadId: string,
file: Blob,
fileSigned: string,
thumbnailBuffer: ArrayBuffer | undefined,
dataKey: CryptoKey,
dataKeyVersion: Date,
) => {
state.status = "uploading";
// The file is cached while uploading, so that it doesn't have to be read again
const cache = createUploadFileCache(file.size);
let fileId;
try {
const { encContentHash } = await uploadBlob(uploadId, file, dataKey, {
onProgress(s) {
state.progress = s.progress;
state.rate = s.rate;
},
onPlaintextChunk: cache.writeChunk,
});
({ file: fileId } = await trpc.upload.completeFileUpload.mutate({
uploadId,
contentHmac: fileSigned,
encContentHash,
}));
} catch (e) {
await cache.discard();
throw e;
}
await cache.commit(fileId).catch(console.error);
if (thumbnailBuffer) {
try {
const { uploadId } = await trpc.upload.startFileThumbnailUpload.mutate({
file: fileId,
dekVersion: dataKeyVersion,
});
await uploadBlob(uploadId, new Blob([thumbnailBuffer]), dataKey);
await trpc.upload.completeFileThumbnailUpload.mutate({ uploadId });
} catch (e) {
console.error(e);
}
}
state.status = "uploaded";
return { fileId };
},
{ concurrency: 1 },
);
export const uploadFile = async (
file: File,
parentId: "root" | number,
onDuplicate: () => Promise<boolean>,
) => {
uploadingFiles.push({
id: crypto.randomUUID(),
name: file.name,
parentId,
status: "queued",
});
const state = uploadingFiles.at(-1)!;
return await scheduler.schedule(file.size, async () => {
try {
await ensureHmacSecrets();
const masterKey = keyring.activeMasterKey;
const hmacSecret = keyring.activeHmacSecret;
const { fileSigned } = await requestDuplicateFileScan(state, file, hmacSecret, onDuplicate);
if (!fileSigned) {
state.status = "canceled";
uploadingFiles = uploadingFiles.filter((file) => file !== state);
return;
}
const fileType = getFileType(file);
const fileMetadata: FileMetadata = {
parentId,
name: file.name,
lastModifiedAt: new Date(file.lastModified),
};
if (fileType.startsWith("image/")) {
fileMetadata.createdAt = await readExifDateTime(file);
}
const blob = new Blob([file], { type: fileType });
const { uploadId, thumbnailBuffer, dataKey, dataKeyVersion } =
await requestFileMetadataEncryption(state, blob, fileMetadata, masterKey, hmacSecret);
const { fileId } = await requestFileUpload(
state,
uploadId,
blob,
fileSigned,
thumbnailBuffer,
dataKey,
dataKeyVersion,
);
return { fileId, thumbnailBuffer };
} catch (e) {
state.status = "error";
throw e;
}
});
};
@@ -1,24 +0,0 @@
<script lang="ts">
import { getFileThumbnail } from "$lib/modules/file";
import type { SummarizedFileInfo } from "$lib/modules/filesystem";
interface Props {
info: SummarizedFileInfo;
onclick?: (file: SummarizedFileInfo) => void;
}
let { info, onclick }: Props = $props();
let thumbnail = $derived(getFileThumbnail(info));
</script>
<button
onclick={onclick && (() => setTimeout(() => onclick(info), 100))}
class="aspect-square overflow-hidden rounded transition active:scale-95 active:brightness-90"
>
{#if $thumbnail}
<img src={$thumbnail} alt={info.name} class="h-full w-full object-cover" />
{:else}
<div class="h-full w-full bg-gray-100"></div>
{/if}
</button>
@@ -1,6 +0,0 @@
export { default as ActionEntryButton } from "./ActionEntryButton.svelte";
export { default as Button } from "./Button.svelte";
export { default as EntryButton } from "./EntryButton.svelte";
export { default as FileThumbnailButton } from "./FileThumbnailButton.svelte";
export { default as FloatingButton } from "./FloatingButton.svelte";
export { default as TextButton } from "./TextButton.svelte";
-3
View File
@@ -1,3 +0,0 @@
export { default as AdaptiveDiv } from "./AdaptiveDiv.svelte";
export { default as BottomDiv } from "./BottomDiv.svelte";
export { default as FullscreenDiv } from "./FullscreenDiv.svelte";
-6
View File
@@ -1,6 +0,0 @@
export { default as BottomSheet } from "./BottomSheet.svelte";
export * from "./buttons";
export * from "./divs";
export * from "./inputs";
export { default as Modal } from "./Modal.svelte";
export { default as RowVirtualizer } from "./RowVirtualizer.svelte";
-2
View File
@@ -1,2 +0,0 @@
export { default as CheckBox } from "./CheckBox.svelte";
export { default as TextInput } from "./TextInput.svelte";
-9
View File
@@ -1,9 +0,0 @@
export * from "./ActionModal.svelte";
export { default as ActionModal } from "./ActionModal.svelte";
export * from "./Categories.svelte";
export { default as Categories } from "./Categories.svelte";
export { default as IconEntryButton } from "./IconEntryButton.svelte";
export * from "./labels";
export { default as SubCategories } from "./SubCategories.svelte";
export { default as TitledDiv } from "./TitledDiv.svelte";
export { default as TopBar } from "./TopBar.svelte";
@@ -1,4 +0,0 @@
export { default as CategoryLabel } from "./CategoryLabel.svelte";
export { default as DirectoryEntryLabel } from "./DirectoryEntryLabel.svelte";
export { default as IconLabel } from "./IconLabel.svelte";
export { default as TitleLabel } from "./TitleLabel.svelte";
-1
View File
@@ -1 +0,0 @@
export * from "./modals";
@@ -1,4 +0,0 @@
export { default as CategoryCreateModal } from "./CategoryCreateModal.svelte";
export { default as ForceLoginModal } from "./ForceLoginModal.svelte";
export { default as RenameModal } from "./RenameModal.svelte";
export { default as TextInputModal } from "./TextInputModal.svelte";
-2
View File
@@ -1,2 +0,0 @@
export * from "./serviceWorker";
export * from "./upload";
-1
View File
@@ -1 +0,0 @@
export const DECRYPTED_FILE_URL_PREFIX = "/_internal/decrypted-file/";
-6
View File
@@ -1,6 +0,0 @@
export const AES_GCM_IV_SIZE = 12;
export const AES_GCM_TAG_SIZE = 16;
export const ENCRYPTION_OVERHEAD = AES_GCM_IV_SIZE + AES_GCM_TAG_SIZE;
export const CHUNK_SIZE = 4 * 1024 * 1024;
export const ENCRYPTED_CHUNK_SIZE = CHUNK_SIZE + ENCRYPTION_OVERHEAD;
@@ -1,5 +1,5 @@
<script lang="ts"> <script lang="ts">
import { ActionModal } from "$lib/components/molecules"; import { ActionModal } from "#lib/ui/index.ts";
interface Props { interface Props {
isOpen: boolean; isOpen: boolean;
@@ -0,0 +1 @@
export { default as ForceLoginModal } from "./ForceLoginModal.svelte";
+13
View File
@@ -0,0 +1,13 @@
import { trpc } from "#trpc/client.ts";
export const login = async (email: string, password: string) => {
await trpc.auth.login.mutate({ email, password });
};
export const logout = async () => {
await trpc.auth.logout.mutate();
};
export const changePassword = async (oldPassword: string, newPassword: string) => {
await trpc.auth.changePassword.mutate({ oldPassword, newPassword });
};
+30
View File
@@ -0,0 +1,30 @@
import { showToast } from "#lib/ui/index.ts";
import { isTRPCClientError } from "#trpc/client.ts";
const getErrorMessage = (e: unknown, fallbackMessage: string) => {
if (isTRPCClientError(e)) {
switch (e.data?.code) {
case "NOT_FOUND":
return "대상을 찾을 수 없어요. 이미 삭제되었을 수 있어요.";
case undefined:
return "서버에 연결할 수 없어요.";
}
}
return fallbackMessage;
};
export const showErrorToast = (e: unknown, fallbackMessage: string) => {
console.error(e);
showToast(getErrorMessage(e, fallbackMessage));
};
// Runs the action and shows the error to the user if it fails. Returns whether it succeeded.
export const tryAction = async (action: () => Promise<unknown>, errorMessage: string) => {
try {
await action();
return true;
} catch (e) {
showErrorToast(e, errorMessage);
return false;
}
};
+54
View File
@@ -0,0 +1,54 @@
import {
getDirectoryInfo,
getFileInfo,
isExistingFileInfo,
isLocalDirectoryInfo,
setDirectoryFavorite,
setFileFavorite,
type LocalDirectoryInfo,
type SummarizedFileInfo,
} from "#lib/features/filesystem/index.ts";
import { sortEntries } from "#lib/utils/index.ts";
import { trpc } from "#trpc/client.ts";
export type FavoriteEntry =
| { type: "directory"; name: string; details: LocalDirectoryInfo }
| { type: "file"; name: string; details: SummarizedFileInfo };
export const getFavoriteEntries = async () => {
const favorites = await trpc.favorites.get.query();
const [directories, files] = await Promise.all([
Promise.all(
favorites.directories.map((directory) =>
getDirectoryInfo(directory.id, { serverResponse: { ...directory, isFavorite: true } }),
),
),
Promise.all(
favorites.files.map((file) =>
getFileInfo(file.id, { serverResponse: { ...file, isFavorite: true } }),
),
),
]);
return [
...sortEntries(
directories.filter(isLocalDirectoryInfo).map((directory): FavoriteEntry => ({
type: "directory",
name: directory.name,
details: directory,
})),
),
...sortEntries(
files
.filter(isExistingFileInfo)
.map((file): FavoriteEntry => ({ type: "file", name: file.name, details: file })),
),
];
};
export const removeFavorite = async ({ type, details }: FavoriteEntry) => {
if (type === "directory") {
await setDirectoryFavorite(details, false);
} else {
await setFileFavorite(details, false);
}
};
+150
View File
@@ -0,0 +1,150 @@
/* eslint-disable svelte/prefer-svelte-reactivity -- Lookup tables of reactive entries */
import { untrack } from "svelte";
interface FilesystemCacheOptions<K, V> {
fetchFromIndexedDB: (key: K) => Promise<V | undefined>;
fetchFromServer: (key: K, cachedValue: V | undefined) => Promise<V>;
bulkFetchFromIndexedDB?: (keys: Set<K>) => Promise<Map<K, V>>;
bulkFetchFromServer?: (keys: Map<K, { cachedValue: V | undefined }>) => Promise<Map<K, V>>;
}
interface Entry<V> {
value?: V;
promise?: Promise<V>;
isInvalidated: boolean;
}
// Caches filesystem entries as reactive objects which are updated in place, so that every view
// of an entry reflects its latest state. Values are first loaded from IndexedDB (if any) and then
// revalidated against the server.
export class FilesystemCache<K, V extends object> {
private map = new Map<K, Entry<V>>();
constructor(private readonly options: FilesystemCacheOptions<K, V>) {}
private getOrCreateEntry(key: K) {
let entry = this.map.get(key);
if (!entry) {
const newEntry: Entry<V> = $state({ isInvalidated: false });
entry = newEntry;
this.map.set(key, entry);
}
return entry;
}
private settle(entry: Entry<V>, value: V) {
if (entry.value) {
Object.assign(entry.value, value);
} else {
entry.value = value;
}
return entry.value!;
}
private finish(key: K, entry: Entry<V>) {
entry.promise = undefined;
// Mutations may invalidate the entry while it is being revalidated, in which case it is queried
// again unless it was evicted or replaced in the meantime
if (entry.isInvalidated && this.map.get(key) === entry) {
void Promise.resolve(this.get(key)).catch(() => {});
}
}
// Returns the cached value synchronously if available, and revalidates it in the background
get(key: K, options?: { fetchFromServer?: (cachedValue: V | undefined) => Promise<V> }) {
return untrack(() => {
const entry = this.getOrCreateEntry(key);
if (entry.promise) return entry.value ?? entry.promise;
const { promise, resolve, reject } = Promise.withResolvers<V>();
promise.catch(() => {}); // Rejections are handled by callers waiting for the promise
entry.promise = promise;
entry.isInvalidated = false;
(async () => {
let cachedValue = entry.value && ($state.snapshot(entry.value) as V);
if (!cachedValue) {
cachedValue = await this.options.fetchFromIndexedDB(key);
if (cachedValue) {
resolve(this.settle(entry, cachedValue));
}
}
const loadedValue = await (options?.fetchFromServer?.(cachedValue) ??
this.options.fetchFromServer(key, cachedValue));
resolve(this.settle(entry, loadedValue));
})()
.catch((e) => {
console.error(e);
reject(e);
})
.finally(() => this.finish(key, entry));
return entry.value ?? promise;
});
}
bulkGet(keys: Set<K>) {
return untrack(() => {
const entries = new Map<K, Entry<V>>();
const pending = new Map<K, PromiseWithResolvers<V>>();
for (const key of keys) {
const entry = this.getOrCreateEntry(key);
entries.set(key, entry);
if (entry.promise) continue;
const resolvers = Promise.withResolvers<V>();
resolvers.promise.catch(() => {});
entry.promise = resolvers.promise;
entry.isInvalidated = false;
pending.set(key, resolvers);
}
const settleAll = (values: Map<K, V>) => {
values.forEach((value, key) => {
pending.get(key)?.resolve(this.settle(entries.get(key)!, value));
});
};
(async () => {
const pendingKeys = [...pending.keys()];
settleAll(
await this.options.bulkFetchFromIndexedDB!(
new Set(pendingKeys.filter((key) => !entries.get(key)!.value)),
),
);
settleAll(
await this.options.bulkFetchFromServer!(
new Map(pendingKeys.map((key) => [key, { cachedValue: entries.get(key)!.value }])),
),
);
})()
.catch((e) => {
console.error(e);
pending.forEach(({ reject }) => reject(e));
})
.finally(() => {
pending.forEach((_, key) => this.finish(key, entries.get(key)!));
});
const makeResult = () => new Map([...entries].map(([key, entry]) => [key, entry.value!]));
const waiting = [...entries.values()]
.filter((entry) => !entry.value)
.map((entry) => entry.promise!);
return waiting.length > 0 ? Promise.all(waiting).then(makeResult) : makeResult();
});
}
// Revalidates the entry if it has been loaded
invalidate(key: K) {
const entry = this.map.get(key);
if (entry) {
entry.isInvalidated = true;
void Promise.resolve(this.get(key)).catch(() => {});
}
}
evict(key: K) {
this.map.delete(key);
}
}
+168
View File
@@ -0,0 +1,168 @@
import * as IndexedDB from "#lib/client/storage/indexed-db/index.ts";
import { isTRPCClientError, trpc } from "#trpc/client.ts";
import { FilesystemCache } from "./cache.svelte.ts";
import { invalidateFile } from "./file.ts";
import { toCategoryInfo, toFileInfo } from "./dto.ts";
import { encryptName, generateEntryKeys } from "./metadata.ts";
import { useResource } from "./resource.svelte.ts";
import type { CategoryInfo, DataKey, LocalCategoryInfo, MaybeCategoryInfo } from "./types.ts";
const cache = new FilesystemCache<CategoryId, MaybeCategoryInfo>({
async fetchFromIndexedDB(id) {
const [category, subCategories] = await Promise.all([
id !== "root" ? IndexedDB.getCategoryInfo(id) : undefined,
IndexedDB.getCategoryInfos(id),
]);
const files = category?.files
? await Promise.all(
category.files.map(async (file) => {
const fileInfo = await IndexedDB.getFileInfo(file.id);
return fileInfo
? {
id: file.id,
parentId: fileInfo.parentId,
contentType: fileInfo.contentType,
name: fileInfo.name,
createdAt: fileInfo.createdAt,
lastModifiedAt: fileInfo.lastModifiedAt,
isFavorite: fileInfo.isFavorite,
isRecursive: file.isRecursive,
}
: undefined;
}),
)
: undefined;
if (id === "root") {
return {
id,
exists: true,
subCategories,
};
} else if (category) {
return {
id,
exists: true,
parentId: category.parentId,
name: category.name,
subCategories,
files: files?.filter((file) => !!file) ?? [],
isFileRecursive: category.isFileRecursive ?? false,
};
}
},
async fetchFromServer(id, cachedInfo) {
try {
const category = await trpc.category.get.query({ id, recurse: true });
const [subCategories, files, metadata] = await Promise.all([
Promise.all(category.subCategories.map(toCategoryInfo)),
Promise.all(
(category.files ?? []).map(async (file) => ({
...(await toFileInfo(file)),
isRecursive: file.isRecursive,
})),
),
category.metadata && toCategoryInfo(category.metadata),
]);
return storeToIndexedDB(
metadata
? {
...metadata,
subCategories,
files,
isFileRecursive: cachedInfo?.isFileRecursive ?? false,
}
: { id: "root", subCategories },
);
} catch (e) {
if (isTRPCClientError(e) && e.data?.code === "NOT_FOUND") {
await IndexedDB.deleteCategoryInfo(id as number);
return { id, exists: false };
}
throw e;
}
},
});
const storeToIndexedDB = (info: CategoryInfo) => {
if (info.id !== "root") {
void IndexedDB.storeCategoryInfo(info);
// TODO: Bulk Upsert
new Map(info.files.map((file) => [file.id, file])).forEach((file) => {
void IndexedDB.storeFileInfo(file);
});
}
// TODO: Bulk Upsert
info.subCategories.forEach((category) => {
void IndexedDB.storeCategoryInfo(category);
});
void IndexedDB.deleteDanglingCategoryInfos(
info.id,
new Set(info.subCategories.map(({ id }) => id)),
);
return { ...info, exists: true as const };
};
export const getCategoryInfo = (id: CategoryId) => {
return cache.get(id);
};
export const useCategory = (getId: () => CategoryId) => {
return useResource(getId, (id) => getCategoryInfo(id));
};
export const invalidateCategory = (id: CategoryId) => {
cache.invalidate(id);
};
export const createCategory = async (parentId: CategoryId, name: string) => {
await trpc.category.create.mutate({ parent: parentId, ...(await generateEntryKeys(name)) });
invalidateCategory(parentId);
};
export const renameCategory = async (
category: { id: number; parentId: CategoryId; dataKey?: DataKey },
newName: string,
) => {
if (!category.dataKey) {
throw new Error("Data key not loaded");
}
await trpc.category.rename.mutate({
id: category.id,
dekVersion: category.dataKey.version,
...(await encryptName(newName, category.dataKey.key)),
});
invalidateCategory(category.parentId);
invalidateCategory(category.id);
};
export const deleteCategory = async (category: { id: number; parentId: CategoryId }) => {
await trpc.category.delete.mutate({ id: category.id });
cache.evict(category.id);
invalidateCategory(category.parentId);
};
export const addFileToCategory = async (fileId: number, categoryId: number) => {
await trpc.category.addFile.mutate({ id: categoryId, file: fileId });
invalidateFile(fileId);
invalidateCategory(categoryId);
};
export const removeFileFromCategory = async (fileId: number, categoryId: number) => {
await trpc.category.removeFile.mutate({ id: categoryId, file: fileId });
invalidateFile(fileId);
invalidateCategory(categoryId);
};
// Whether to show the files of subcategories is a local preference
export const setCategoryFileRecursive = async (category: LocalCategoryInfo, value: boolean) => {
category.isFileRecursive = value;
await IndexedDB.updateCategoryInfo(category.id, { isFileRecursive: value });
};
@@ -1,5 +1,5 @@
<script module lang="ts"> <script module lang="ts">
import type { DataKey } from "$lib/modules/filesystem"; import type { DataKey, SubCategoryInfo } from "#lib/features/filesystem/index.ts";
export interface SelectedCategory { export interface SelectedCategory {
id: number; id: number;
@@ -11,10 +11,8 @@
<script lang="ts"> <script lang="ts">
import type { Component } from "svelte"; import type { Component } from "svelte";
import type { SvelteHTMLElements } from "svelte/elements"; import type { SvelteHTMLElements } from "svelte/elements";
import { ActionEntryButton } from "$lib/components/atoms"; import { ActionEntryButton, CategoryLabel } from "#lib/ui/index.ts";
import { CategoryLabel } from "$lib/components/molecules"; import { sortEntries } from "#lib/utils/index.ts";
import type { SubCategoryInfo } from "$lib/modules/filesystem";
import { sortEntries } from "$lib/utils";
interface Props { interface Props {
categories: SubCategoryInfo[]; categories: SubCategoryInfo[];
@@ -1,5 +1,5 @@
<script lang="ts"> <script lang="ts">
import { TextInputModal } from "$lib/components/organisms"; import { TextInputModal } from "#lib/ui/index.ts";
interface Props { interface Props {
isOpen: boolean; isOpen: boolean;
@@ -0,0 +1,25 @@
<script lang="ts">
import type { Component } from "svelte";
import type { SvelteHTMLElements } from "svelte/elements";
import { ActionEntryButton, DirectoryEntryLabel } from "#lib/ui/index.ts";
interface Props {
action?: {
icon: Component<SvelteHTMLElements["svg"]>;
onclick: () => void;
};
info: { name: string; isFavorite?: boolean };
onclick?: () => void;
}
let { action, info, onclick }: Props = $props();
</script>
<ActionEntryButton
class="h-14"
{onclick}
actionButtonIcon={action?.icon}
onActionButtonClick={action?.onclick}
>
<DirectoryEntryLabel type="directory" name={info.name} isFavorite={info.isFavorite} />
</ActionEntryButton>
@@ -0,0 +1,40 @@
<script lang="ts">
import type { Component } from "svelte";
import type { ClassValue, SvelteHTMLElements } from "svelte/elements";
import { getFileThumbnail } from "#lib/client/cache/index.ts";
import { ActionEntryButton, DirectoryEntryLabel } from "#lib/ui/index.ts";
import { formatDateTime } from "#lib/utils/index.ts";
import type { SummarizedFileInfo } from "../types.ts";
interface Props {
action?: {
icon: Component<SvelteHTMLElements["svg"]>;
onclick: () => void;
class?: ClassValue;
};
info: SummarizedFileInfo;
onclick?: () => void;
showThumbnail?: boolean;
subtext?: string;
}
let { action, info, onclick, showThumbnail = true, subtext }: Props = $props();
let thumbnail = $derived(showThumbnail ? getFileThumbnail(info) : undefined);
</script>
<ActionEntryButton
class="h-14"
{onclick}
actionButtonIcon={action?.icon}
actionButtonClass={action?.class}
onActionButtonClick={action?.onclick}
>
<DirectoryEntryLabel
type="file"
thumbnail={thumbnail?.url}
name={info.name}
subtext={subtext ?? formatDateTime(info.createdAt ?? info.lastModifiedAt)}
isFavorite={info.isFavorite}
/>
</ActionEntryButton>
@@ -0,0 +1,35 @@
<script lang="ts">
import { getFileThumbnail } from "#lib/client/cache/index.ts";
import type { SummarizedFileInfo } from "#lib/features/filesystem/index.ts";
import { afterPress } from "#lib/ui/index.ts";
import IconFavorite from "~icons/material-symbols/favorite";
interface Props {
info: SummarizedFileInfo;
onclick?: (file: SummarizedFileInfo) => void;
}
let { info, onclick }: Props = $props();
let thumbnail = $derived(getFileThumbnail(info));
</script>
<button
onclick={afterPress(onclick && (() => onclick(info)))}
class="relative aspect-square overflow-hidden rounded-sm transition active:scale-95 active:brightness-90"
>
{#if thumbnail?.url}
<img src={thumbnail?.url} alt={info.name} class="size-full object-cover" />
{:else}
<div class="size-full bg-gray-100"></div>
{/if}
{#if info.isFavorite}
<div class={["absolute right-0.5 bottom-0.5"]}>
<IconFavorite
class="text-sm text-red-500"
style="filter: drop-shadow(0 0 1px white) drop-shadow(0 0 1px white)"
/>
</div>
{/if}
</button>
@@ -1,8 +1,9 @@
<script lang="ts"> <script lang="ts">
import type { Component } from "svelte"; import type { Component } from "svelte";
import type { ClassValue, SvelteHTMLElements } from "svelte/elements"; import type { ClassValue, SvelteHTMLElements } from "svelte/elements";
import { Categories, IconEntryButton, type SelectedCategory } from "$lib/components/molecules"; import { Categories, type SelectedCategory } from "#lib/features/filesystem/components/index.ts";
import type { CategoryInfo } from "$lib/modules/filesystem"; import type { CategoryInfo } from "#lib/features/filesystem/index.ts";
import { IconEntryButton } from "#lib/ui/index.ts";
import IconAddCircle from "~icons/material-symbols/add-circle"; import IconAddCircle from "~icons/material-symbols/add-circle";
@@ -10,9 +11,9 @@
class?: ClassValue; class?: ClassValue;
info: CategoryInfo; info: CategoryInfo;
onSubCategoryClick: (subCategory: SelectedCategory) => void; onSubCategoryClick: (subCategory: SelectedCategory) => void;
onSubCategoryCreateClick: () => void; onSubCategoryCreateClick?: () => void;
onSubCategoryMenuClick?: (category: SelectedCategory) => void; onSubCategoryMenuClick?: (category: SelectedCategory) => void;
subCategoryCreatePosition?: "top" | "bottom"; subCategoryCreatePosition?: "top" | "bottom" | "none";
subCategoryMenuIcon?: Component<SvelteHTMLElements["svg"]>; subCategoryMenuIcon?: Component<SvelteHTMLElements["svg"]>;
} }
@@ -22,7 +23,7 @@
onSubCategoryClick, onSubCategoryClick,
onSubCategoryCreateClick, onSubCategoryCreateClick,
onSubCategoryMenuClick, onSubCategoryMenuClick,
subCategoryCreatePosition = "bottom", subCategoryCreatePosition = "none",
subCategoryMenuIcon, subCategoryMenuIcon,
}: Props = $props(); }: Props = $props();
</script> </script>
@@ -0,0 +1,7 @@
export * from "./Categories.svelte";
export { default as Categories } from "./Categories.svelte";
export { default as CategoryCreateModal } from "./CategoryCreateModal.svelte";
export { default as DirectoryEntry } from "./DirectoryEntry.svelte";
export { default as FileEntry } from "./FileEntry.svelte";
export { default as FileThumbnailButton } from "./FileThumbnailButton.svelte";
export { default as SubCategories } from "./SubCategories.svelte";
+67
View File
@@ -0,0 +1,67 @@
import {
deleteFileCache,
deleteFileThumbnailCache,
storeFileThumbnailCache,
} from "#lib/client/cache/index.ts";
import {
getDecryptedFileUrl,
prepareFileDecryption,
saveDecryptedFile,
} from "#lib/client/service-worker.ts";
import { getAllFileInfos } from "#lib/client/storage/indexed-db/index.ts";
import { uploadBlob } from "#lib/client/transfer/chunked-upload.ts";
import { trpc } from "#trpc/client.ts";
import type { DataKey } from "./types.ts";
interface StreamableFile {
id: number;
isLegacy?: boolean;
dataKey?: DataKey;
contentType: string;
}
// Returns a URL from which the service worker streams the decrypted content of the file
export const getFileStreamUrl = async ({ id, isLegacy, dataKey, contentType }: StreamableFile) => {
if (!dataKey) {
throw new Error("Data key not loaded");
}
await prepareFileDecryption(id, { isLegacy: !!isLegacy, dataKey: dataKey.key, contentType });
return getDecryptedFileUrl(id);
};
export const getFileBlob = async (file: StreamableFile) => {
const res = await fetch(await getFileStreamUrl(file));
if (!res.ok) {
throw new Error("Failed to download the file");
}
return await res.blob();
};
export const saveFile = async (file: StreamableFile & { name: string }) => {
await getFileStreamUrl(file);
saveDecryptedFile(file.id, file.name);
};
export const uploadFileThumbnail = async (fileId: number, thumbnail: Blob, dataKey: DataKey) => {
const { uploadId } = await trpc.upload.startFileThumbnailUpload.mutate({
file: fileId,
dekVersion: dataKey.version,
});
await uploadBlob(uploadId, thumbnail, dataKey.key);
await trpc.upload.completeFileThumbnailUpload.mutate({ uploadId });
await storeFileThumbnailCache(fileId, await thumbnail.arrayBuffer());
};
// Deletes local caches of files deleted on other devices
export const cleanupDeletedFileCaches = async () => {
const liveFiles = new Set(await trpc.file.list.query());
const maybeCachedFiles = await getAllFileInfos();
await Promise.all(
maybeCachedFiles
.filter(({ id }) => !liveFiles.has(id))
.flatMap(({ id }) => [deleteFileCache(id), deleteFileThumbnailCache(id)]),
);
};
+151
View File
@@ -0,0 +1,151 @@
import { deleteFileCache, deleteFileThumbnailCache } from "#lib/client/cache/index.ts";
import * as IndexedDB from "#lib/client/storage/indexed-db/index.ts";
import { isTRPCClientError, trpc } from "#trpc/client.ts";
import { FilesystemCache } from "./cache.svelte.ts";
import { toDirectoryInfo, toFileInfo, type DirectoryDto } from "./dto.ts";
import { encryptName, generateEntryKeys } from "./metadata.ts";
import { useResource } from "./resource.svelte.ts";
import type { DataKey, DirectoryInfo, MaybeDirectoryInfo } from "./types.ts";
const cache = new FilesystemCache<DirectoryId, MaybeDirectoryInfo>({
async fetchFromIndexedDB(id) {
const [directory, subDirectories, files] = await Promise.all([
id !== "root" ? IndexedDB.getDirectoryInfo(id) : undefined,
IndexedDB.getDirectoryInfos(id),
IndexedDB.getFileInfos(id),
]);
if (id === "root") {
return {
id,
exists: true,
subDirectories,
files,
};
} else if (directory) {
return {
id,
exists: true,
parentId: directory.parentId,
name: directory.name,
subDirectories,
files,
isFavorite: directory.isFavorite,
};
}
},
async fetchFromServer(id) {
try {
const directory = await trpc.directory.get.query({ id });
const [subDirectories, files, metadata] = await Promise.all([
Promise.all(directory.subDirectories.map(toDirectoryInfo)),
Promise.all(directory.files.map(toFileInfo)),
directory.metadata && toDirectoryInfo(directory.metadata),
]);
return storeToIndexedDB(
metadata ? { ...metadata, subDirectories, files } : { id: "root", subDirectories, files },
);
} catch (e) {
if (isTRPCClientError(e) && e.data?.code === "NOT_FOUND") {
await IndexedDB.deleteDirectoryInfo(id as number);
return { id, exists: false as const };
}
throw e;
}
},
});
const storeToIndexedDB = (info: DirectoryInfo) => {
if (info.id !== "root") {
void IndexedDB.storeDirectoryInfo(info);
}
// TODO: Bulk Upsert
info.subDirectories.forEach((subDirectory) => {
void IndexedDB.storeDirectoryInfo(subDirectory);
});
// TODO: Bulk Upsert
info.files.forEach((file) => {
void IndexedDB.storeFileInfo(file);
});
void IndexedDB.deleteDanglingDirectoryInfos(
info.id,
new Set(info.subDirectories.map(({ id }) => id)),
);
void IndexedDB.deleteDanglingFileInfos(info.id, new Set(info.files.map(({ id }) => id)));
return { ...info, exists: true as const };
};
// Uses `serverResponse` instead of fetching the directory from the server if given
export const getDirectoryInfo = (id: DirectoryId, options?: { serverResponse?: DirectoryDto }) => {
const serverResponse = options?.serverResponse;
return cache.get(id, {
fetchFromServer:
serverResponse &&
(async (cachedValue) =>
storeToIndexedDB({
subDirectories: [],
files: [],
...cachedValue,
...(await toDirectoryInfo(serverResponse)),
})),
});
};
export const useDirectory = (getId: () => DirectoryId) => {
return useResource(getId, (id) => getDirectoryInfo(id));
};
export const invalidateDirectory = (id: DirectoryId) => {
cache.invalidate(id);
};
export const createDirectory = async (parentId: DirectoryId, name: string) => {
await trpc.directory.create.mutate({ parent: parentId, ...(await generateEntryKeys(name)) });
invalidateDirectory(parentId);
};
export const renameDirectory = async (
directory: { id: number; parentId: DirectoryId; dataKey?: DataKey },
newName: string,
) => {
if (!directory.dataKey) {
throw new Error("Data key not loaded");
}
await trpc.directory.rename.mutate({
id: directory.id,
dekVersion: directory.dataKey.version,
...(await encryptName(newName, directory.dataKey.key)),
});
invalidateDirectory(directory.parentId);
invalidateDirectory(directory.id);
};
export const deleteDirectory = async (directory: { id: number; parentId: DirectoryId }) => {
const { deletedFiles } = await trpc.directory.delete.mutate({ id: directory.id });
cache.evict(directory.id);
invalidateDirectory(directory.parentId);
await Promise.all(
deletedFiles.flatMap((fileId) => [deleteFileCache(fileId), deleteFileThumbnailCache(fileId)]),
);
};
export const setDirectoryFavorite = async (
directory: { id: number; parentId: DirectoryId },
isFavorite: boolean,
) => {
if (isFavorite) {
await trpc.favorites.addDirectory.mutate({ id: directory.id });
} else {
await trpc.favorites.removeDirectory.mutate({ id: directory.id });
}
invalidateDirectory(directory.parentId);
invalidateDirectory(directory.id);
};
+46
View File
@@ -0,0 +1,46 @@
import type { RouterOutputs } from "#trpc/router.server.ts";
import {
decryptCategoryMetadata,
decryptDirectoryMetadata,
decryptFileMetadata,
} from "./metadata.ts";
import type { FileInfo, SubCategoryInfo, SubDirectoryInfo, SummarizedFileInfo } from "./types.ts";
// Shapes of filesystem entries returned by the server
export type DirectoryDto = RouterOutputs["directory"]["get"]["subDirectories"][number];
export type FileDto = RouterOutputs["directory"]["get"]["files"][number];
export type CategoryDto = RouterOutputs["category"]["get"]["subCategories"][number];
export type FileWithCategoriesDto = RouterOutputs["file"]["get"];
export const toDirectoryInfo = async (directory: DirectoryDto): Promise<SubDirectoryInfo> => ({
id: directory.id,
parentId: directory.parent,
isFavorite: directory.isFavorite,
...(await decryptDirectoryMetadata(directory)),
});
export const toFileInfo = async (file: FileDto): Promise<SummarizedFileInfo> => ({
id: file.id,
isLegacy: file.isLegacy,
parentId: file.parent,
contentType: file.contentType,
isFavorite: file.isFavorite,
...(await decryptFileMetadata(file)),
});
export const toCategoryInfo = async (category: CategoryDto): Promise<SubCategoryInfo> => ({
id: category.id,
parentId: category.parent,
...(await decryptCategoryMetadata(category)),
});
export const toFileWithCategoriesInfo = async ({
categories,
...file
}: FileWithCategoriesDto): Promise<FileInfo> => {
const [fileInfo, categoryInfos] = await Promise.all([
toFileInfo(file),
Promise.all(categories.map(toCategoryInfo)),
]);
return { ...fileInfo, categories: categoryInfos };
};
+196
View File
@@ -0,0 +1,196 @@
import { deleteFileCache, deleteFileThumbnailCache } from "#lib/client/cache/index.ts";
import * as IndexedDB from "#lib/client/storage/indexed-db/index.ts";
import { isTRPCClientError, trpc } from "#trpc/client.ts";
import { FilesystemCache } from "./cache.svelte.ts";
import { invalidateDirectory } from "./directory.ts";
import { toFileInfo, toFileWithCategoriesInfo, type FileDto } from "./dto.ts";
import { encryptName } from "./metadata.ts";
import { useResource } from "./resource.svelte.ts";
import type { DataKey, FileInfo, MaybeFileInfo } from "./types.ts";
// Categories of files stored in IndexedDB, which may have been deleted
const getFileCategoryInfos = async (categoryIds: number[] | undefined) => {
const categories = await Promise.all(
(categoryIds ?? []).map((id) => IndexedDB.getCategoryInfo(id)),
);
return categories
.filter((category) => !!category)
.map(({ id, parentId, name }) => ({ id, parentId, name }));
};
const cache = new FilesystemCache<number, MaybeFileInfo>({
async fetchFromIndexedDB(id) {
const file = await IndexedDB.getFileInfo(id);
if (file) {
return {
id,
exists: true,
parentId: file.parentId,
contentType: file.contentType,
name: file.name,
createdAt: file.createdAt,
lastModifiedAt: file.lastModifiedAt,
categories: await getFileCategoryInfos(file.categoryIds),
isFavorite: file.isFavorite,
};
}
},
async fetchFromServer(id) {
try {
return storeToIndexedDB(await toFileWithCategoriesInfo(await trpc.file.get.query({ id })));
} catch (e) {
if (isTRPCClientError(e) && e.data?.code === "NOT_FOUND") {
await IndexedDB.deleteFileInfo(id);
return { id, exists: false as const };
}
throw e;
}
},
async bulkFetchFromIndexedDB(ids) {
const files = (await IndexedDB.bulkGetFileInfos([...ids])).filter((file) => !!file);
return new Map(
await Promise.all(
files.map(
async (file) =>
[
file.id,
{
...file,
exists: true as const,
categories: await getFileCategoryInfos(file.categoryIds),
},
] as const,
),
),
);
},
async bulkFetchFromServer(ids) {
const idsArray = [...ids.keys()];
const filesRaw = await trpc.file.bulkGet.query({ ids: idsArray });
const files = await Promise.all(filesRaw.map(toFileWithCategoriesInfo));
const existingIds = new Set(filesRaw.map(({ id }) => id));
const deletedIds = idsArray.filter((id) => !existingIds.has(id));
void IndexedDB.bulkDeleteFileInfos(deletedIds);
return new Map<number, MaybeFileInfo>([
...bulkStoreToIndexedDB(files),
...deletedIds.map((id) => [id, { id, exists: false }] as const),
]);
},
});
const storeToIndexedDB = (info: FileInfo) => {
void IndexedDB.storeFileInfo({
...info,
categoryIds: info.categories.map(({ id }) => id),
});
info.categories.forEach((category) => {
void IndexedDB.storeCategoryInfo(category);
});
return { ...info, exists: true as const };
};
const bulkStoreToIndexedDB = (infos: FileInfo[]) => {
// TODO: Bulk Upsert
infos.forEach((info) => {
void IndexedDB.storeFileInfo({
...info,
categoryIds: info.categories.map(({ id }) => id),
});
});
// TODO: Bulk Upsert
new Map(
infos.flatMap(({ categories }) => categories).map((category) => [category.id, category]),
).forEach((category) => {
void IndexedDB.storeCategoryInfo(category);
});
return infos.map((info) => [info.id, { ...info, exists: true }] as const);
};
// Uses `serverResponse` instead of fetching the file from the server if given
export const getFileInfo = (id: number, options?: { serverResponse?: FileDto }) => {
const serverResponse = options?.serverResponse;
return cache.get(id, {
fetchFromServer:
serverResponse &&
(async (cachedValue) =>
storeToIndexedDB({
categories: [],
...cachedValue,
...(await toFileInfo(serverResponse)),
})),
});
};
export const bulkGetFileInfo = (ids: number[]) => {
return cache.bulkGet(new Set(ids));
};
export const useFile = (getId: () => number) => {
return useResource(getId, (id) => getFileInfo(id));
};
export const invalidateFile = (id: number) => {
cache.invalidate(id);
};
export const renameFile = async (
file: { id: number; parentId: DirectoryId; dataKey?: DataKey },
newName: string,
) => {
if (!file.dataKey) {
throw new Error("Data key not loaded");
}
await trpc.file.rename.mutate({
id: file.id,
dekVersion: file.dataKey.version,
...(await encryptName(newName, file.dataKey.key)),
});
invalidateDirectory(file.parentId);
invalidateFile(file.id);
};
export const deleteFile = async (file: { id: number; parentId: DirectoryId }) => {
await trpc.file.delete.mutate({ id: file.id });
cache.evict(file.id);
invalidateDirectory(file.parentId);
await Promise.all([deleteFileCache(file.id), deleteFileThumbnailCache(file.id)]);
};
export const setFileFavorite = async (
file: { id: number; parentId: DirectoryId },
isFavorite: boolean,
) => {
if (isFavorite) {
await trpc.favorites.addFile.mutate({ id: file.id });
} else {
await trpc.favorites.removeFile.mutate({ id: file.id });
}
invalidateDirectory(file.parentId);
invalidateFile(file.id);
};
// The most recently created media files among the files stored locally
export const getRecentMediaFileInfos = async (limit: number) => {
const files = await IndexedDB.getAllFileInfos();
return files
.filter(
({ contentType }) => contentType.startsWith("image/") || contentType.startsWith("video/"),
)
.sort(
(a, b) =>
(b.createdAt ?? b.lastModifiedAt).getTime() - (a.createdAt ?? a.lastModifiedAt).getTime(),
)
.slice(0, limit);
};
+6
View File
@@ -0,0 +1,6 @@
export * from "./category.ts";
export * from "./content.ts";
export * from "./directory.ts";
export * from "./file.ts";
export * from "./types.ts";
export * from "./upload.ts";
+83
View File
@@ -0,0 +1,83 @@
import { keyring } from "#lib/client/keys/index.ts";
import {
decryptString,
encryptString,
generateDataKey,
unwrapDataKey,
wrapDataKey,
} from "#lib/shared/crypto/index.ts";
const unwrapEntryDataKey = async (metadata: { mekVersion: number; dek: string }) => {
const { dataKey } = await unwrapDataKey(
metadata.dek,
keyring.getMasterKey(metadata.mekVersion).key,
);
return dataKey;
};
export const decryptDirectoryMetadata = async (metadata: {
mekVersion: number;
dek: string;
dekVersion: Date;
name: string;
nameIv: string;
}) => {
const dataKey = await unwrapEntryDataKey(metadata);
const name = await decryptString(metadata.name, metadata.nameIv, dataKey);
return {
dataKey: { key: dataKey, version: metadata.dekVersion },
name,
};
};
const decryptDate = async (ciphertext: string, iv: string, dataKey: CryptoKey) => {
return new Date(parseInt(await decryptString(ciphertext, iv, dataKey), 10));
};
export const decryptFileMetadata = async (metadata: {
mekVersion: number;
dek: string;
dekVersion: Date;
name: string;
nameIv: string;
createdAt?: string;
createdAtIv?: string;
lastModifiedAt: string;
lastModifiedAtIv: string;
}) => {
const dataKey = await unwrapEntryDataKey(metadata);
const [name, createdAt, lastModifiedAt] = await Promise.all([
decryptString(metadata.name, metadata.nameIv, dataKey),
metadata.createdAt
? decryptDate(metadata.createdAt, metadata.createdAtIv!, dataKey)
: undefined,
decryptDate(metadata.lastModifiedAt, metadata.lastModifiedAtIv, dataKey),
]);
return {
dataKey: { key: dataKey, version: metadata.dekVersion },
name,
createdAt,
lastModifiedAt,
};
};
export const decryptCategoryMetadata = decryptDirectoryMetadata;
export const encryptName = async (name: string, dataKey: CryptoKey) => {
const { ciphertext, iv } = await encryptString(name, dataKey);
return { name: ciphertext, nameIv: iv };
};
// Generates the keys of a new directory or category encrypted with the active master key
export const generateEntryKeys = async (name: string) => {
const masterKey = keyring.activeMasterKey;
const { dataKey, dataKeyVersion } = await generateDataKey();
return {
mekVersion: masterKey.version,
dek: await wrapDataKey(dataKey, masterKey.key),
dekVersion: dataKeyVersion,
...(await encryptName(name, dataKey)),
};
};
@@ -0,0 +1,33 @@
import { untrack } from "svelte";
// Exposes the value of a cache entry identified by a reactive key. Cached values are available
// synchronously, so that views don't flicker while revalidating.
export const useResource = <K, V>(getKey: () => K, load: (key: K) => V | Promise<V>) => {
let current: V | undefined = $state();
$effect.pre(() => {
const key = getKey();
let isStale = false;
const result = untrack(() => load(key));
if (result instanceof Promise) {
result
.then((value) => {
if (!isStale) current = value;
})
.catch(() => {}); // Logged by the cache
} else {
current = result;
}
return () => {
isStale = true;
};
});
return {
get current() {
return current;
},
};
};
@@ -1,22 +1,23 @@
export type DataKey = { key: CryptoKey; version: Date }; export type DataKey = { key: CryptoKey; version: Date };
type AllUndefined<T> = { [K in keyof T]?: undefined };
interface LocalDirectoryInfo { export interface LocalDirectoryInfo {
id: number; id: number;
parentId: DirectoryId; parentId: DirectoryId;
dataKey?: DataKey; dataKey?: DataKey;
name: string; name: string;
subDirectories: SubDirectoryInfo[]; subDirectories: SubDirectoryInfo[];
files: SummarizedFileInfo[]; files: SummarizedFileInfo[];
isFavorite: boolean;
} }
interface RootDirectoryInfo { export interface RootDirectoryInfo {
id: "root"; id: "root";
parentId?: undefined; parentId?: undefined;
dataKey?: undefined; dataKey?: undefined;
name?: undefined; name?: undefined;
subDirectories: SubDirectoryInfo[]; subDirectories: SubDirectoryInfo[];
files: SummarizedFileInfo[]; files: SummarizedFileInfo[];
isFavorite?: undefined;
} }
export type DirectoryInfo = LocalDirectoryInfo | RootDirectoryInfo; export type DirectoryInfo = LocalDirectoryInfo | RootDirectoryInfo;
@@ -36,6 +37,7 @@ export interface FileInfo {
createdAt?: Date; createdAt?: Date;
lastModifiedAt: Date; lastModifiedAt: Date;
categories: FileCategoryInfo[]; categories: FileCategoryInfo[];
isFavorite: boolean;
} }
export type MaybeFileInfo = export type MaybeFileInfo =
@@ -45,9 +47,9 @@ export type MaybeFileInfo =
export type SummarizedFileInfo = Omit<FileInfo, "categories">; export type SummarizedFileInfo = Omit<FileInfo, "categories">;
export type CategoryFileInfo = SummarizedFileInfo & { isRecursive: boolean }; export type CategoryFileInfo = SummarizedFileInfo & { isRecursive: boolean };
interface LocalCategoryInfo { export interface LocalCategoryInfo {
id: number; id: number;
parentId: DirectoryId; parentId: CategoryId;
dataKey?: DataKey; dataKey?: DataKey;
name: string; name: string;
subCategories: SubCategoryInfo[]; subCategories: SubCategoryInfo[];
@@ -55,7 +57,7 @@ interface LocalCategoryInfo {
isFileRecursive: boolean; isFileRecursive: boolean;
} }
interface RootCategoryInfo { export interface RootCategoryInfo {
id: "root"; id: "root";
parentId?: undefined; parentId?: undefined;
dataKey?: undefined; dataKey?: undefined;
@@ -75,3 +77,10 @@ export type SubCategoryInfo = Omit<
"subCategories" | "files" | "isFileRecursive" "subCategories" | "files" | "isFileRecursive"
>; >;
export type FileCategoryInfo = Omit<SubCategoryInfo, "dataKey">; export type FileCategoryInfo = Omit<SubCategoryInfo, "dataKey">;
export const isLocalDirectoryInfo = (
info: MaybeDirectoryInfo,
): info is LocalDirectoryInfo & { exists: true } => info.exists && info.id !== "root";
export const isExistingFileInfo = (info: MaybeFileInfo): info is FileInfo & { exists: true } =>
info.exists;
+19
View File
@@ -0,0 +1,19 @@
import { storeFileThumbnailCache } from "#lib/client/cache/index.ts";
import { uploadFile as doUploadFile } from "#lib/client/transfer/index.ts";
import { invalidateDirectory } from "./directory.ts";
// Uploads the file and caches it with its thumbnail locally. Returns `false` if the upload is canceled.
export const uploadFile = async (
file: File,
parentId: DirectoryId,
onDuplicate: () => Promise<boolean>,
) => {
const res = await doUploadFile(file, parentId, onDuplicate);
if (!res) return false;
invalidateDirectory(parentId);
if (res.thumbnailBuffer) {
void storeFileThumbnailCache(res.fileId, res.thumbnailBuffer);
}
return true;
};
+46
View File
@@ -0,0 +1,46 @@
import {
getDirectoryInfo,
getFileInfo,
isExistingFileInfo,
isLocalDirectoryInfo,
type FileInfo,
type LocalCategoryInfo,
type LocalDirectoryInfo,
} from "#lib/features/filesystem/index.ts";
import { trpc } from "#trpc/client.ts";
export interface SearchFilter {
ancestorId: DirectoryId;
inFavorites: boolean;
categories: { info: LocalCategoryInfo; type: "include" | "exclude" }[];
}
export interface SearchResult {
directories: LocalDirectoryInfo[];
files: FileInfo[];
}
const getCategoryIds = (filter: SearchFilter, type: "include" | "exclude") => {
return filter.categories.filter((category) => category.type === type).map(({ info }) => info.id);
};
export const search = async (filter: SearchFilter) => {
const { directories: directoriesRaw, files: filesRaw } = await trpc.search.search.query({
ancestor: filter.ancestorId,
inFavorites: filter.inFavorites,
includeCategories: getCategoryIds(filter, "include"),
excludeCategories: getCategoryIds(filter, "exclude"),
});
const [directories, files] = await Promise.all([
Promise.all(
directoriesRaw.map((directory) =>
getDirectoryInfo(directory.id, { serverResponse: directory }),
),
),
Promise.all(filesRaw.map((file) => getFileInfo(file.id, { serverResponse: file }))),
]);
return {
directories: directories.filter(isLocalDirectoryInfo),
files: files.filter(isExistingFileInfo),
} satisfies SearchResult;
};
@@ -0,0 +1,12 @@
<script lang="ts">
import { formatNetworkSpeed } from "#lib/utils/index.ts";
interface Props {
progress?: number;
rate?: number; // Bytes per second
}
let { progress, rate }: Props = $props();
</script>
전송됨 {Math.floor((progress ?? 0) * 100)}% · {formatNetworkSpeed((rate ?? 0) * 8)}
@@ -0,0 +1,26 @@
<script lang="ts">
import type { FileUploadState } from "#lib/client/transfer/index.ts";
import TransferProgress from "./TransferProgress.svelte";
interface Props {
state: FileUploadState;
}
let { state }: Props = $props();
const statusTexts = {
queued: "대기 중",
"encryption-pending": "준비 중",
encrypting: "암호화하는 중",
"upload-pending": "업로드를 기다리는 중",
uploaded: "업로드 완료",
canceled: "",
error: "업로드 실패",
} satisfies Record<Exclude<FileUploadState["status"], "uploading">, string>;
</script>
{#if state.status === "uploading"}
<TransferProgress progress={state.progress} rate={state.rate} />
{:else}
{statusTexts[state.status]}
{/if}
@@ -0,0 +1,2 @@
export { default as TransferProgress } from "./TransferProgress.svelte";
export { default as UploadStatusText } from "./UploadStatusText.svelte";
-28
View File
@@ -1,28 +0,0 @@
import { Dexie, type EntityTable } from "dexie";
export interface FileCacheIndex {
fileId: number;
cachedAt: Date;
lastRetrievedAt: Date;
size: number;
}
const cacheIndex = new Dexie("cacheIndex") as Dexie & {
fileCache: EntityTable<FileCacheIndex, "fileId">;
};
cacheIndex.version(1).stores({
fileCache: "fileId",
});
export const getFileCacheIndex = async () => {
return await cacheIndex.fileCache.toArray();
};
export const storeFileCacheIndex = async (fileCacheIndex: FileCacheIndex) => {
await cacheIndex.fileCache.put(fileCacheIndex);
};
export const deleteFileCacheIndex = async (fileId: number) => {
await cacheIndex.fileCache.delete(fileId);
};
-3
View File
@@ -1,3 +0,0 @@
export * from "./cacheIndex";
export * from "./filesystem";
export * from "./keyStore";
-4
View File
@@ -1,4 +0,0 @@
export * from "./aes";
export * from "./rsa";
export * from "./sha";
export * from "./util";
-50
View File
@@ -1,50 +0,0 @@
import {
getFileCacheIndex as getFileCacheIndexFromIndexedDB,
storeFileCacheIndex,
deleteFileCacheIndex,
type FileCacheIndex,
} from "$lib/indexedDB";
import { readFile, writeFile, deleteFile } from "$lib/modules/opfs";
const fileCacheIndex = new Map<number, FileCacheIndex>();
export const prepareFileCache = async () => {
for (const cache of await getFileCacheIndexFromIndexedDB()) {
fileCacheIndex.set(cache.fileId, cache);
}
};
export const getFileCacheIndex = () => {
return Array.from(fileCacheIndex.values());
};
export const getFileCache = async (fileId: number) => {
const cacheIndex = fileCacheIndex.get(fileId);
if (!cacheIndex) return null;
cacheIndex.lastRetrievedAt = new Date();
storeFileCacheIndex(cacheIndex); // Intended
return await readFile(`/cache/${fileId}`);
};
export const storeFileCache = async (fileId: number, fileBuffer: ArrayBuffer) => {
const now = new Date();
await writeFile(`/cache/${fileId}`, fileBuffer);
const cacheIndex: FileCacheIndex = {
fileId,
cachedAt: now,
lastRetrievedAt: now,
size: fileBuffer.byteLength,
};
fileCacheIndex.set(fileId, cacheIndex);
await storeFileCacheIndex(cacheIndex);
};
export const deleteFileCache = async (fileId: number) => {
if (!fileCacheIndex.has(fileId)) return;
fileCacheIndex.delete(fileId);
await deleteFile(`/cache/${fileId}`);
await deleteFileCacheIndex(fileId);
};
-110
View File
@@ -1,110 +0,0 @@
import axios from "axios";
import { limitFunction } from "p-limit";
import { CHUNK_SIZE, ENCRYPTION_OVERHEAD } from "$lib/constants";
import { decryptChunk, concatenateBuffers } from "$lib/modules/crypto";
export interface FileDownloadState {
id: number;
status:
| "download-pending"
| "downloading"
| "decryption-pending"
| "decrypting"
| "decrypted"
| "canceled"
| "error";
progress?: number;
rate?: number;
estimated?: number;
result?: ArrayBuffer;
}
type LiveFileDownloadState = FileDownloadState & {
status: "download-pending" | "downloading" | "decryption-pending" | "decrypting";
};
let downloadingFiles: FileDownloadState[] = $state([]);
export const isFileDownloading = (
status: FileDownloadState["status"],
): status is LiveFileDownloadState["status"] =>
["download-pending", "downloading", "decryption-pending", "decrypting"].includes(status);
export const getFileDownloadState = (fileId: number) => {
return downloadingFiles.find((file) => file.id === fileId && isFileDownloading(file.status));
};
export const getDownloadingFiles = () => {
return downloadingFiles.filter((file) => isFileDownloading(file.status));
};
export const clearDownloadedFiles = () => {
downloadingFiles = downloadingFiles.filter((file) => isFileDownloading(file.status));
};
const requestFileDownload = limitFunction(
async (state: FileDownloadState, id: number) => {
state.status = "downloading";
const res = await axios.get(`/api/file/${id}/download`, {
responseType: "arraybuffer",
onDownloadProgress: ({ progress, rate, estimated }) => {
state.progress = progress;
state.rate = rate;
state.estimated = estimated;
},
});
const fileEncrypted: ArrayBuffer = res.data;
state.status = "decryption-pending";
return fileEncrypted;
},
{ concurrency: 1 },
);
const decryptFile = limitFunction(
async (
state: FileDownloadState,
fileEncrypted: ArrayBuffer,
encryptedChunkSize: number,
dataKey: CryptoKey,
) => {
state.status = "decrypting";
const chunks: ArrayBuffer[] = [];
let offset = 0;
while (offset < fileEncrypted.byteLength) {
const nextOffset = Math.min(offset + encryptedChunkSize, fileEncrypted.byteLength);
chunks.push(await decryptChunk(fileEncrypted.slice(offset, nextOffset), dataKey));
offset = nextOffset;
}
const fileBuffer = concatenateBuffers(...chunks).buffer;
state.status = "decrypted";
state.result = fileBuffer;
return fileBuffer;
},
{ concurrency: 4 },
);
export const downloadFile = async (id: number, dataKey: CryptoKey, isLegacy: boolean) => {
downloadingFiles.push({
id,
status: "download-pending",
});
const state = downloadingFiles.at(-1)!;
try {
const fileEncrypted = await requestFileDownload(state, id);
return await decryptFile(
state,
fileEncrypted,
isLegacy ? fileEncrypted.byteLength : CHUNK_SIZE + ENCRYPTION_OVERHEAD,
dataKey,
);
} catch (e) {
state.status = "error";
throw e;
}
};
-4
View File
@@ -1,4 +0,0 @@
export * from "./cache";
export * from "./download.svelte";
export * from "./thumbnail";
export * from "./upload.svelte";
-82
View File
@@ -1,82 +0,0 @@
import { LRUCache } from "lru-cache";
import { writable, type Writable } from "svelte/store";
import { browser } from "$app/environment";
import { decryptData } from "$lib/modules/crypto";
import type { SummarizedFileInfo } from "$lib/modules/filesystem";
import { readFile, writeFile, deleteFile, deleteDirectory } from "$lib/modules/opfs";
import { getThumbnailUrl } from "$lib/modules/thumbnail";
const loadedThumbnails = new LRUCache<number, Writable<string>>({ max: 100 });
const loadingThumbnails = new Map<number, Writable<string | undefined>>();
const fetchFromOpfs = async (fileId: number) => {
const thumbnailBuffer = await readFile(`/thumbnail/file/${fileId}`);
if (thumbnailBuffer) {
return getThumbnailUrl(thumbnailBuffer);
}
};
const fetchFromServer = async (fileId: number, dataKey: CryptoKey) => {
const res = await fetch(`/api/file/${fileId}/thumbnail/download`);
if (!res.ok) return null;
const thumbnailEncrypted = await res.arrayBuffer();
const thumbnailBuffer = await decryptData(
thumbnailEncrypted.slice(12),
thumbnailEncrypted.slice(0, 12),
dataKey,
);
void writeFile(`/thumbnail/file/${fileId}`, thumbnailBuffer);
return getThumbnailUrl(thumbnailBuffer);
};
export const getFileThumbnail = (file: SummarizedFileInfo) => {
if (
!browser ||
!(file.contentType.startsWith("image/") || file.contentType.startsWith("video/"))
) {
return undefined;
}
const thumbnail = loadedThumbnails.get(file.id);
if (thumbnail) return thumbnail;
let loadingThumbnail = loadingThumbnails.get(file.id);
if (loadingThumbnail) return loadingThumbnail;
loadingThumbnail = writable(undefined);
loadingThumbnails.set(file.id, loadingThumbnail);
fetchFromOpfs(file.id)
.then((thumbnail) => thumbnail ?? (file.dataKey && fetchFromServer(file.id, file.dataKey.key)))
.then((thumbnail) => {
if (thumbnail) {
loadingThumbnail.set(thumbnail);
loadedThumbnails.set(file.id, loadingThumbnail as Writable<string>);
}
loadingThumbnails.delete(file.id);
});
return loadingThumbnail;
};
export const storeFileThumbnailCache = async (fileId: number, thumbnailBuffer: ArrayBuffer) => {
await writeFile(`/thumbnail/file/${fileId}`, thumbnailBuffer);
const oldThumbnail = loadedThumbnails.get(fileId);
if (oldThumbnail) {
oldThumbnail.set(getThumbnailUrl(thumbnailBuffer));
} else {
loadedThumbnails.set(fileId, writable(getThumbnailUrl(thumbnailBuffer)));
}
};
export const deleteFileThumbnailCache = async (fileId: number) => {
loadedThumbnails.delete(fileId);
await deleteFile(`/thumbnail/file/${fileId}`);
};
export const deleteAllFileThumbnailCaches = async () => {
loadedThumbnails.clear();
await deleteDirectory(`/thumbnail/file`);
};
-320
View File
@@ -1,320 +0,0 @@
import axios from "axios";
import ExifReader from "exifreader";
import { limitFunction } from "p-limit";
import { CHUNK_SIZE } from "$lib/constants";
import {
encodeToBase64,
generateDataKey,
wrapDataKey,
encryptData,
encryptString,
encryptChunk,
digestMessage,
signMessageHmac,
} from "$lib/modules/crypto";
import { Scheduler } from "$lib/modules/scheduler";
import { generateThumbnail } from "$lib/modules/thumbnail";
import type { FileThumbnailUploadRequest } from "$lib/server/schemas";
import type { MasterKey, HmacSecret } from "$lib/stores";
import { trpc } from "$trpc/client";
import type { RouterInputs } from "$trpc/router.server";
export interface FileUploadState {
name: string;
parentId: DirectoryId;
status:
| "queued"
| "encryption-pending"
| "encrypting"
| "upload-pending"
| "uploading"
| "uploaded"
| "canceled"
| "error";
progress?: number;
rate?: number;
estimated?: number;
}
export type LiveFileUploadState = FileUploadState & {
status: "queued" | "encryption-pending" | "encrypting" | "upload-pending" | "uploading";
};
const scheduler = new Scheduler<
{ fileId: number; fileBuffer: ArrayBuffer; thumbnailBuffer?: ArrayBuffer } | undefined
>();
let uploadingFiles: FileUploadState[] = $state([]);
const isFileUploading = (status: FileUploadState["status"]) =>
["queued", "encryption-pending", "encrypting", "upload-pending", "uploading"].includes(status);
export const getUploadingFiles = (parentId?: DirectoryId) => {
return uploadingFiles.filter(
(file) =>
(parentId === undefined || file.parentId === parentId) && isFileUploading(file.status),
);
};
export const clearUploadedFiles = () => {
uploadingFiles = uploadingFiles.filter((file) => isFileUploading(file.status));
};
const requestDuplicateFileScan = limitFunction(
async (file: File, hmacSecret: HmacSecret, onDuplicate: () => Promise<boolean>) => {
const fileBuffer = await file.arrayBuffer();
const fileSigned = encodeToBase64(await signMessageHmac(fileBuffer, hmacSecret.secret));
const files = await trpc().file.listByHash.query({
hskVersion: hmacSecret.version,
contentHmac: fileSigned,
});
if (files.length === 0 || (await onDuplicate())) {
return { fileBuffer, fileSigned };
} else {
return {};
}
},
{ concurrency: 1 },
);
const getFileType = (file: File) => {
if (file.type) return file.type;
if (file.name.endsWith(".heic")) return "image/heic";
throw new Error("Unknown file type");
};
const extractExifDateTime = (fileBuffer: ArrayBuffer) => {
const exif = ExifReader.load(fileBuffer);
const dateTimeOriginal = exif["DateTimeOriginal"]?.description;
const offsetTimeOriginal = exif["OffsetTimeOriginal"]?.description;
if (!dateTimeOriginal) return undefined;
const [date, time] = dateTimeOriginal.split(" ");
if (!date || !time) return undefined;
const [year, month, day] = date.split(":").map(Number);
const [hour, minute, second] = time.split(":").map(Number);
if (!year || !month || !day || !hour || !minute || !second) return undefined;
if (!offsetTimeOriginal) {
// No timezone information.. Assume local timezone
return new Date(year, month - 1, day, hour, minute, second);
}
const offsetSign = offsetTimeOriginal[0] === "+" ? 1 : -1;
const [offsetHour, offsetMinute] = offsetTimeOriginal.slice(1).split(":").map(Number);
const utcDate = Date.UTC(year, month - 1, day, hour, minute, second);
const offsetMs = offsetSign * ((offsetHour ?? 0) * 60 + (offsetMinute ?? 0)) * 60 * 1000;
return new Date(utcDate - offsetMs);
};
const encryptChunks = async (fileBuffer: ArrayBuffer, dataKey: CryptoKey) => {
const chunksEncrypted: { chunkEncrypted: ArrayBuffer; chunkEncryptedHash: string }[] = [];
let offset = 0;
while (offset < fileBuffer.byteLength) {
const nextOffset = Math.min(offset + CHUNK_SIZE, fileBuffer.byteLength);
const chunkEncrypted = await encryptChunk(fileBuffer.slice(offset, nextOffset), dataKey);
chunksEncrypted.push({
chunkEncrypted: chunkEncrypted,
chunkEncryptedHash: encodeToBase64(await digestMessage(chunkEncrypted)),
});
offset = nextOffset;
}
return chunksEncrypted;
};
const encryptFile = limitFunction(
async (state: FileUploadState, file: File, fileBuffer: ArrayBuffer, masterKey: MasterKey) => {
state.status = "encrypting";
const fileType = getFileType(file);
let createdAt;
if (fileType.startsWith("image/")) {
createdAt = extractExifDateTime(fileBuffer);
}
const { dataKey, dataKeyVersion } = await generateDataKey();
const dataKeyWrapped = await wrapDataKey(dataKey, masterKey.key);
const chunksEncrypted = await encryptChunks(fileBuffer, dataKey);
const nameEncrypted = await encryptString(file.name, dataKey);
const createdAtEncrypted =
createdAt && (await encryptString(createdAt.getTime().toString(), dataKey));
const lastModifiedAtEncrypted = await encryptString(file.lastModified.toString(), dataKey);
const thumbnail = await generateThumbnail(fileBuffer, fileType);
const thumbnailBuffer = await thumbnail?.arrayBuffer();
const thumbnailEncrypted = thumbnailBuffer && (await encryptData(thumbnailBuffer, dataKey));
state.status = "upload-pending";
return {
dataKeyWrapped,
dataKeyVersion,
fileType,
chunksEncrypted,
nameEncrypted,
createdAtEncrypted,
lastModifiedAtEncrypted,
thumbnail: thumbnailEncrypted && { plaintext: thumbnailBuffer, ...thumbnailEncrypted },
};
},
{ concurrency: 4 },
);
const requestFileUpload = limitFunction(
async (
state: FileUploadState,
metadata: RouterInputs["file"]["startUpload"],
chunksEncrypted: { chunkEncrypted: ArrayBuffer; chunkEncryptedHash: string }[],
fileSigned: string | undefined,
thumbnailForm: FormData | null,
) => {
state.status = "uploading";
const { uploadId } = await trpc().file.startUpload.mutate(metadata);
// Upload chunks with progress tracking
const totalBytes = chunksEncrypted.reduce((sum, c) => sum + c.chunkEncrypted.byteLength, 0);
let uploadedBytes = 0;
const startTime = Date.now();
for (let i = 0; i < chunksEncrypted.length; i++) {
const { chunkEncrypted, chunkEncryptedHash } = chunksEncrypted[i]!;
const response = await fetch(`/api/file/upload/${uploadId}/chunks/${i}`, {
method: "POST",
headers: {
"Content-Type": "application/octet-stream",
"Content-Digest": `sha-256=:${chunkEncryptedHash}:`,
},
body: chunkEncrypted,
});
if (!response.ok) {
throw new Error(`Chunk upload failed: ${response.status} ${response.statusText}`);
}
uploadedBytes += chunkEncrypted.byteLength;
// Calculate progress, rate, estimated
const elapsed = (Date.now() - startTime) / 1000; // seconds
const rate = uploadedBytes / elapsed; // bytes per second
const remaining = totalBytes - uploadedBytes;
const estimated = rate > 0 ? remaining / rate : undefined;
state.progress = uploadedBytes / totalBytes;
state.rate = rate;
state.estimated = estimated;
}
// Complete upload
const { file: fileId } = await trpc().file.completeUpload.mutate({
uploadId,
contentHmac: fileSigned,
});
// Upload thumbnail if exists
if (thumbnailForm) {
try {
await axios.post(`/api/file/${fileId}/thumbnail/upload`, thumbnailForm);
} catch (e) {
// TODO: Error handling for thumbnail upload
console.error(e);
}
}
state.status = "uploaded";
return { fileId };
},
{ concurrency: 1 },
);
export const uploadFile = async (
file: File,
parentId: "root" | number,
hmacSecret: HmacSecret,
masterKey: MasterKey,
onDuplicate: () => Promise<boolean>,
) => {
uploadingFiles.push({
name: file.name,
parentId,
status: "queued",
});
const state = uploadingFiles.at(-1)!;
return await scheduler.schedule(file.size, async () => {
state.status = "encryption-pending";
try {
const { fileBuffer, fileSigned } = await requestDuplicateFileScan(
file,
hmacSecret,
onDuplicate,
);
if (!fileBuffer || !fileSigned) {
state.status = "canceled";
uploadingFiles = uploadingFiles.filter((file) => file !== state);
return undefined;
}
const {
dataKeyWrapped,
dataKeyVersion,
fileType,
chunksEncrypted,
nameEncrypted,
createdAtEncrypted,
lastModifiedAtEncrypted,
thumbnail,
} = await encryptFile(state, file, fileBuffer, masterKey);
const metadata = {
chunks: chunksEncrypted.length,
parent: parentId,
mekVersion: masterKey.version,
dek: dataKeyWrapped,
dekVersion: dataKeyVersion,
hskVersion: hmacSecret.version,
contentType: fileType,
name: nameEncrypted.ciphertext,
nameIv: nameEncrypted.iv,
createdAt: createdAtEncrypted?.ciphertext,
createdAtIv: createdAtEncrypted?.iv,
lastModifiedAt: lastModifiedAtEncrypted.ciphertext,
lastModifiedAtIv: lastModifiedAtEncrypted.iv,
};
let thumbnailForm = null;
if (thumbnail) {
thumbnailForm = new FormData();
thumbnailForm.set(
"metadata",
JSON.stringify({
dekVersion: dataKeyVersion.toISOString(),
contentIv: encodeToBase64(thumbnail.iv),
} satisfies FileThumbnailUploadRequest),
);
thumbnailForm.set("content", new Blob([thumbnail.ciphertext]));
}
const { fileId } = await requestFileUpload(
state,
metadata,
chunksEncrypted,
fileSigned,
thumbnailForm,
);
return { fileId, fileBuffer, thumbnailBuffer: thumbnail?.plaintext };
} catch (e) {
state.status = "error";
throw e;
}
});
};
-121
View File
@@ -1,121 +0,0 @@
import * as IndexedDB from "$lib/indexedDB";
import { trpc, isTRPCClientError } from "$trpc/client";
import { FilesystemCache, decryptFileMetadata, decryptCategoryMetadata } from "./internal.svelte";
import type { CategoryInfo, MaybeCategoryInfo } from "./types";
const cache = new FilesystemCache<CategoryId, MaybeCategoryInfo>({
async fetchFromIndexedDB(id) {
const [category, subCategories] = await Promise.all([
id !== "root" ? IndexedDB.getCategoryInfo(id) : undefined,
IndexedDB.getCategoryInfos(id),
]);
const files = category?.files
? await Promise.all(
category.files.map(async (file) => {
const fileInfo = await IndexedDB.getFileInfo(file.id);
return fileInfo
? {
id: file.id,
parentId: fileInfo.parentId,
contentType: fileInfo.contentType,
name: fileInfo.name,
createdAt: fileInfo.createdAt,
lastModifiedAt: fileInfo.lastModifiedAt,
isRecursive: file.isRecursive,
}
: undefined;
}),
)
: undefined;
if (id === "root") {
return {
id,
exists: true,
subCategories,
};
} else if (category) {
return {
id,
exists: true,
parentId: category.parentId,
name: category.name,
subCategories,
files: files?.filter((file) => !!file) ?? [],
isFileRecursive: category.isFileRecursive ?? false,
};
}
},
async fetchFromServer(id, cachedInfo, masterKey) {
try {
const category = await trpc().category.get.query({ id, recurse: true });
const [subCategories, files, metadata] = await Promise.all([
Promise.all(
category.subCategories.map(async (category) => ({
id: category.id,
parentId: id,
...(await decryptCategoryMetadata(category, masterKey)),
})),
),
category.files &&
Promise.all(
category.files.map(async (file) => ({
id: file.id,
parentId: file.parent,
contentType: file.contentType,
isRecursive: file.isRecursive,
...(await decryptFileMetadata(file, masterKey)),
})),
),
category.metadata && decryptCategoryMetadata(category.metadata, masterKey),
]);
return storeToIndexedDB(
id !== "root"
? {
id,
parentId: category.metadata!.parent,
subCategories,
files: files!,
isFileRecursive: cachedInfo?.isFileRecursive ?? false,
...metadata!,
}
: { id, subCategories },
);
} catch (e) {
if (isTRPCClientError(e) && e.data?.code === "NOT_FOUND") {
await IndexedDB.deleteCategoryInfo(id as number);
return { id, exists: false };
}
throw e;
}
},
});
const storeToIndexedDB = (info: CategoryInfo) => {
if (info.id !== "root") {
void IndexedDB.storeCategoryInfo(info);
// TODO: Bulk Upsert
new Map(info.files.map((file) => [file.id, file])).forEach((file) => {
void IndexedDB.storeFileInfo(file);
});
}
// TODO: Bulk Upsert
info.subCategories.forEach((category) => {
void IndexedDB.storeCategoryInfo(category);
});
void IndexedDB.deleteDanglingCategoryInfos(
info.id,
new Set(info.subCategories.map(({ id }) => id)),
);
return { ...info, exists: true as const };
};
export const getCategoryInfo = (id: CategoryId, masterKey: CryptoKey) => {
return cache.get(id, masterKey);
};
-102
View File
@@ -1,102 +0,0 @@
import * as IndexedDB from "$lib/indexedDB";
import { trpc, isTRPCClientError } from "$trpc/client";
import { FilesystemCache, decryptDirectoryMetadata, decryptFileMetadata } from "./internal.svelte";
import type { DirectoryInfo, MaybeDirectoryInfo } from "./types";
const cache = new FilesystemCache<DirectoryId, MaybeDirectoryInfo>({
async fetchFromIndexedDB(id) {
const [directory, subDirectories, files] = await Promise.all([
id !== "root" ? IndexedDB.getDirectoryInfo(id) : undefined,
IndexedDB.getDirectoryInfos(id),
IndexedDB.getFileInfos(id),
]);
if (id === "root") {
return {
id,
exists: true,
subDirectories,
files,
};
} else if (directory) {
return {
id,
exists: true,
parentId: directory.parentId,
name: directory.name,
subDirectories,
files,
};
}
},
async fetchFromServer(id, _cachedInfo, masterKey) {
try {
const directory = await trpc().directory.get.query({ id });
const [subDirectories, files, metadata] = await Promise.all([
Promise.all(
directory.subDirectories.map(async (directory) => ({
id: directory.id,
parentId: id,
...(await decryptDirectoryMetadata(directory, masterKey)),
})),
),
Promise.all(
directory.files.map(async (file) => ({
id: file.id,
parentId: id,
contentType: file.contentType,
...(await decryptFileMetadata(file, masterKey)),
})),
),
directory.metadata && decryptDirectoryMetadata(directory.metadata, masterKey),
]);
return storeToIndexedDB(
id !== "root"
? {
id,
parentId: directory.metadata!.parent,
subDirectories,
files,
...metadata!,
}
: { id, subDirectories, files },
);
} catch (e) {
if (isTRPCClientError(e) && e.data?.code === "NOT_FOUND") {
await IndexedDB.deleteDirectoryInfo(id as number);
return { id, exists: false as const };
}
throw e;
}
},
});
const storeToIndexedDB = (info: DirectoryInfo) => {
if (info.id !== "root") {
void IndexedDB.storeDirectoryInfo(info);
}
// TODO: Bulk Upsert
info.subDirectories.forEach((subDirectory) => {
void IndexedDB.storeDirectoryInfo(subDirectory);
});
// TODO: Bulk Upsert
info.files.forEach((file) => {
void IndexedDB.storeFileInfo(file);
});
void IndexedDB.deleteDanglingDirectoryInfos(
info.id,
new Set(info.subDirectories.map(({ id }) => id)),
);
void IndexedDB.deleteDanglingFileInfos(info.id, new Set(info.files.map(({ id }) => id)));
return { ...info, exists: true as const };
};
export const getDirectoryInfo = (id: DirectoryId, masterKey: CryptoKey) => {
return cache.get(id, masterKey);
};
-177
View File
@@ -1,177 +0,0 @@
import * as IndexedDB from "$lib/indexedDB";
import { trpc, isTRPCClientError } from "$trpc/client";
import { FilesystemCache, decryptFileMetadata, decryptCategoryMetadata } from "./internal.svelte";
import type { FileInfo, MaybeFileInfo } from "./types";
const cache = new FilesystemCache<number, MaybeFileInfo>({
async fetchFromIndexedDB(id) {
const file = await IndexedDB.getFileInfo(id);
const categories = file?.categoryIds
? await Promise.all(
file.categoryIds.map(async (categoryId) => {
const category = await IndexedDB.getCategoryInfo(categoryId);
return category
? { id: category.id, parentId: category.parentId, name: category.name }
: undefined;
}),
)
: undefined;
if (file) {
return {
id,
exists: true,
parentId: file.parentId,
contentType: file.contentType,
name: file.name,
createdAt: file.createdAt,
lastModifiedAt: file.lastModifiedAt,
categories: categories?.filter((category) => !!category) ?? [],
};
}
},
async fetchFromServer(id, _cachedInfo, masterKey) {
try {
const file = await trpc().file.get.query({ id });
const [categories, metadata] = await Promise.all([
Promise.all(
file.categories.map(async (category) => ({
id: category.id,
parentId: category.parent,
...(await decryptCategoryMetadata(category, masterKey)),
})),
),
decryptFileMetadata(file, masterKey),
]);
return storeToIndexedDB({
id,
isLegacy: file.isLegacy,
parentId: file.parent,
dataKey: metadata.dataKey,
contentType: file.contentType,
name: metadata.name,
createdAt: metadata.createdAt,
lastModifiedAt: metadata.lastModifiedAt,
categories,
});
} catch (e) {
if (isTRPCClientError(e) && e.data?.code === "NOT_FOUND") {
await IndexedDB.deleteFileInfo(id);
return { id, exists: false as const };
}
throw e;
}
},
async bulkFetchFromIndexedDB(ids) {
const files = await IndexedDB.bulkGetFileInfos([...ids]);
const categories = await Promise.all(
files.map(async (file) =>
file?.categoryIds
? await Promise.all(
file.categoryIds.map(async (categoryId) => {
const category = await IndexedDB.getCategoryInfo(categoryId);
return category
? { id: category.id, parentId: category.parentId, name: category.name }
: undefined;
}),
)
: undefined,
),
);
return new Map(
files
.filter((file) => !!file)
.map((file, index) => [
file.id,
{
...file,
exists: true,
categories: categories[index]?.filter((category) => !!category) ?? [],
},
]),
);
},
async bulkFetchFromServer(ids, masterKey) {
const idsArray = [...ids.keys()];
const filesRaw = await trpc().file.bulkGet.query({ ids: idsArray });
const files = await Promise.all(
filesRaw.map(async ({ id, categories: categoriesRaw, ...metadataRaw }) => {
const [categories, metadata] = await Promise.all([
Promise.all(
categoriesRaw.map(async (category) => ({
id: category.id,
parentId: category.parent,
...(await decryptCategoryMetadata(category, masterKey)),
})),
),
decryptFileMetadata(metadataRaw, masterKey),
]);
return {
id,
exists: true as const,
isLegacy: metadataRaw.isLegacy,
parentId: metadataRaw.parent,
contentType: metadataRaw.contentType,
categories,
...metadata,
};
}),
);
const existingIds = new Set(filesRaw.map(({ id }) => id));
const deletedIds = idsArray.filter((id) => !existingIds.has(id));
void IndexedDB.bulkDeleteFileInfos(deletedIds);
return new Map<number, MaybeFileInfo>([
...bulkStoreToIndexedDB(files),
...deletedIds.map((id) => [id, { id, exists: false }] as const),
]);
},
});
const storeToIndexedDB = (info: FileInfo) => {
void IndexedDB.storeFileInfo({
...info,
categoryIds: info.categories.map(({ id }) => id),
});
info.categories.forEach((category) => {
void IndexedDB.storeCategoryInfo(category);
});
return { ...info, exists: true as const };
};
const bulkStoreToIndexedDB = (infos: FileInfo[]) => {
// TODO: Bulk Upsert
infos.forEach((info) => {
void IndexedDB.storeFileInfo({
...info,
categoryIds: info.categories.map(({ id }) => id),
});
});
// TODO: Bulk Upsert
new Map(
infos.flatMap(({ categories }) => categories).map((category) => [category.id, category]),
).forEach((category) => {
void IndexedDB.storeCategoryInfo(category);
});
return infos.map((info) => [info.id, { ...info, exists: true }] as const);
};
export const getFileInfo = (id: number, masterKey: CryptoKey) => {
return cache.get(id, masterKey);
};
export const bulkGetFileInfo = (ids: number[], masterKey: CryptoKey) => {
return cache.bulkGet(new Set(ids), masterKey);
};
-4
View File
@@ -1,4 +0,0 @@
export * from "./category";
export * from "./directory";
export * from "./file";
export * from "./types";
@@ -1,172 +0,0 @@
import { untrack } from "svelte";
import { unwrapDataKey, decryptString } from "$lib/modules/crypto";
interface FilesystemCacheOptions<K, V> {
fetchFromIndexedDB: (key: K) => Promise<V | undefined>;
fetchFromServer: (key: K, cachedValue: V | undefined, masterKey: CryptoKey) => Promise<V>;
bulkFetchFromIndexedDB?: (keys: Set<K>) => Promise<Map<K, V>>;
bulkFetchFromServer?: (
keys: Map<K, { cachedValue: V | undefined }>,
masterKey: CryptoKey,
) => Promise<Map<K, V>>;
}
export class FilesystemCache<K, V extends object> {
private map = new Map<K, { value?: V; promise?: Promise<V> }>();
constructor(private readonly options: FilesystemCacheOptions<K, V>) {}
get(key: K, masterKey: CryptoKey) {
return untrack(() => {
let state = this.map.get(key);
if (state?.promise) return state.value ?? state.promise;
const { promise: newPromise, resolve } = Promise.withResolvers<V>();
if (!state) {
const newState = $state({});
state = newState;
this.map.set(key, newState);
}
(state.value
? Promise.resolve(state.value)
: this.options.fetchFromIndexedDB(key).then((loadedInfo) => {
if (loadedInfo) {
state.value = loadedInfo;
resolve(state.value);
}
return loadedInfo;
})
)
.then((cachedInfo) => this.options.fetchFromServer(key, cachedInfo, masterKey))
.then((loadedInfo) => {
if (state.value) {
Object.assign(state.value, loadedInfo);
} else {
state.value = loadedInfo;
}
resolve(state.value);
})
.finally(() => {
state.promise = undefined;
});
state.promise = newPromise;
return state.value ?? newPromise;
});
}
bulkGet(keys: Set<K>, masterKey: CryptoKey) {
return untrack(() => {
const newPromises = new Map(
keys
.keys()
.filter((key) => this.map.get(key)?.promise === undefined)
.map((key) => [key, Promise.withResolvers<V>()]),
);
newPromises.forEach(({ promise }, key) => {
const state = this.map.get(key);
if (state) {
state.promise = promise;
} else {
const newState = $state({ promise });
this.map.set(key, newState);
}
});
const resolve = (loadedInfos: Map<K, V>) => {
loadedInfos.forEach((loadedInfo, key) => {
const state = this.map.get(key)!;
if (state.value) {
Object.assign(state.value, loadedInfo);
} else {
state.value = loadedInfo;
}
newPromises.get(key)!.resolve(state.value);
});
return loadedInfos;
};
this.options.bulkFetchFromIndexedDB!(
new Set(newPromises.keys().filter((key) => this.map.get(key)!.value === undefined)),
)
.then(resolve)
.then(() =>
this.options.bulkFetchFromServer!(
new Map(
newPromises.keys().map((key) => [key, { cachedValue: this.map.get(key)!.value }]),
),
masterKey,
),
)
.then(resolve)
.finally(() => {
newPromises.forEach((_, key) => {
this.map.get(key)!.promise = undefined;
});
});
const bottleneckPromises = Array.from(
keys
.keys()
.filter((key) => this.map.get(key)!.value === undefined)
.map((key) => this.map.get(key)!.promise!),
);
const makeResult = () =>
new Map(keys.keys().map((key) => [key, this.map.get(key)!.value!] as const));
return bottleneckPromises.length > 0
? Promise.all(bottleneckPromises).then(makeResult)
: makeResult();
});
}
}
export const decryptDirectoryMetadata = async (
metadata: { dek: string; dekVersion: Date; name: string; nameIv: string },
masterKey: CryptoKey,
) => {
const { dataKey } = await unwrapDataKey(metadata.dek, masterKey);
const name = await decryptString(metadata.name, metadata.nameIv, dataKey);
return {
dataKey: { key: dataKey, version: metadata.dekVersion },
name,
};
};
const decryptDate = async (ciphertext: string, iv: string, dataKey: CryptoKey) => {
return new Date(parseInt(await decryptString(ciphertext, iv, dataKey), 10));
};
export const decryptFileMetadata = async (
metadata: {
dek: string;
dekVersion: Date;
name: string;
nameIv: string;
createdAt?: string;
createdAtIv?: string;
lastModifiedAt: string;
lastModifiedAtIv: string;
},
masterKey: CryptoKey,
) => {
const { dataKey } = await unwrapDataKey(metadata.dek, masterKey);
const [name, createdAt, lastModifiedAt] = await Promise.all([
decryptString(metadata.name, metadata.nameIv, dataKey),
metadata.createdAt
? decryptDate(metadata.createdAt, metadata.createdAtIv!, dataKey)
: undefined,
decryptDate(metadata.lastModifiedAt, metadata.lastModifiedAtIv, dataKey),
]);
return {
dataKey: { key: dataKey, version: metadata.dekVersion },
name,
createdAt,
lastModifiedAt,
};
};
export const decryptCategoryMetadata = decryptDirectoryMetadata;
-14
View File
@@ -1,14 +0,0 @@
export const parseRangeHeader = (rangeHeader: string | null) => {
if (!rangeHeader) return undefined;
const firstRange = rangeHeader.split(",")[0]!.trim();
const parts = firstRange.replace(/bytes=/, "").split("-");
return {
start: parts[0] ? parseInt(parts[0], 10) : undefined,
end: parts[1] ? parseInt(parts[1], 10) : undefined,
};
};
export const getContentRangeHeader = (range?: { start: number; end: number; total: number }) => {
return range && { "Content-Range": `bytes ${range.start}-${range.end}/${range.total}` };
};
+112
View File
@@ -0,0 +1,112 @@
import { ClientRepo, SessionRepo, db } from "#lib/server/db/index.ts";
import env from "#lib/server/config.ts";
import { verifySessionId } from "#lib/server/crypto.ts";
import { AppError, invalidSessionError } from "#lib/server/errors.ts";
export interface Session {
sessionId: string;
userId: number;
clientId?: number;
}
export interface ClientSession extends Session {
clientId: number;
}
export type SessionPermission =
"any" | "notClient" | "anyClient" | "pendingClient" | "activeClient";
export class AuthenticationError extends Error {
constructor(
public status: 400 | 401,
message: string,
) {
super(message);
this.name = "AuthenticationError";
}
}
export const cookieOptions = {
path: "/",
maxAge: env.session.exp / 1000,
secure: true,
sameSite: "strict",
} as const;
export const authenticate = async (sessionIdSigned: string, ip: string, userAgent: string) => {
const sessionId = verifySessionId(sessionIdSigned, env.session.secret);
if (!sessionId) {
throw new AuthenticationError(400, "Invalid session id");
}
const session = await SessionRepo.refreshSession(db, sessionId, ip, userAgent);
if (!session) {
throw new AuthenticationError(401, "Invalid session id");
}
return {
id: sessionId,
userId: session.userId,
clientId: session.clientId ?? undefined,
};
};
const ensureUserClientState = async (
userId: number,
clientId: number | undefined,
state: "pending" | "active",
) => {
if (!clientId) {
throw new AppError("FORBIDDEN", "Forbidden");
}
const userClient = await ClientRepo.getUserClient(db, userId, clientId);
if (!userClient) {
throw invalidSessionError();
} else if (userClient.state !== state) {
throw new AppError("FORBIDDEN", "Forbidden");
}
};
export async function authorize(
locals: App.Locals,
requiredPermission: "any" | "notClient",
): Promise<Session>;
export async function authorize(
locals: App.Locals,
requiredPermission: "anyClient" | "pendingClient" | "activeClient",
): Promise<ClientSession>;
export async function authorize(
locals: App.Locals,
requiredPermission: SessionPermission,
): Promise<Session> {
if (!locals.session) {
throw new AppError("UNAUTHORIZED", "Unauthenticated");
}
const { id: sessionId, userId, clientId } = locals.session;
switch (requiredPermission) {
case "any":
break;
case "notClient":
if (clientId) {
throw new AppError("FORBIDDEN", "Forbidden");
}
break;
case "anyClient":
if (!clientId) {
throw new AppError("FORBIDDEN", "Forbidden");
}
break;
case "pendingClient":
await ensureUserClientState(userId, clientId, "pending");
break;
case "activeClient":
await ensureUserClientState(userId, clientId, "active");
break;
}
return { sessionId, userId, clientId };
}
+35
View File
@@ -0,0 +1,35 @@
import {
NODE_ENV,
DATABASE_HOST,
DATABASE_PORT,
DATABASE_USER,
DATABASE_PASSWORD,
DATABASE_NAME,
SESSION_SECRET,
SESSION_EXPIRES,
USER_CLIENT_CHALLENGE_EXPIRES,
SESSION_UPGRADE_CHALLENGE_EXPIRES,
LIBRARY_PATH,
THUMBNAILS_PATH,
} from "$app/env/private";
export default {
nodeEnv: NODE_ENV,
database: {
host: DATABASE_HOST,
port: DATABASE_PORT,
user: DATABASE_USER,
password: DATABASE_PASSWORD,
name: DATABASE_NAME,
},
session: {
secret: SESSION_SECRET,
exp: SESSION_EXPIRES,
},
challenge: {
userClientExp: USER_CLIENT_CHALLENGE_EXPIRES,
sessionUpgradeExp: SESSION_UPGRADE_CHALLENGE_EXPIRES,
},
libraryPath: LIBRARY_PATH,
thumbnailsPath: THUMBNAILS_PATH,
};
@@ -56,12 +56,13 @@ export const issueSessionId = async (length: number, secret: string) => {
export const verifySessionId = (sessionIdSigned: string, secret: string) => { export const verifySessionId = (sessionIdSigned: string, secret: string) => {
const [sessionIdHex, sessionIdHmac] = sessionIdSigned.split("."); const [sessionIdHex, sessionIdHmac] = sessionIdSigned.split(".");
if (!sessionIdHex || !sessionIdHmac) return; if (!sessionIdHex || !sessionIdHmac) return;
if (
timingSafeEqual( const actualHmac = Buffer.from(sessionIdHmac, "hex");
Buffer.from(sessionIdHmac, "hex"), const expectedHmac = createHmac("sha256", secret)
createHmac("sha256", secret).update(Buffer.from(sessionIdHex, "hex")).digest(), .update(Buffer.from(sessionIdHex, "hex"))
) .digest();
) { // `timingSafeEqual` throws if the lengths differ (e.g. malformed hex)
if (actualHmac.length === expectedHmac.length && timingSafeEqual(actualHmac, expectedHmac)) {
return sessionIdHex; return sessionIdHex;
} }
}; };
-145
View File
@@ -1,145 +0,0 @@
import { IntegrityError } from "./error";
import db from "./kysely";
import type { Ciphertext } from "./schema";
interface Category {
id: number;
parentId: CategoryId;
userId: number;
mekVersion: number;
encDek: string;
dekVersion: Date;
encName: Ciphertext;
}
export type NewCategory = Omit<Category, "id">;
export const registerCategory = async (params: NewCategory) => {
await db.transaction().execute(async (trx) => {
const mek = await trx
.selectFrom("master_encryption_key")
.select("version")
.where("user_id", "=", params.userId)
.where("state", "=", "active")
.limit(1)
.forUpdate()
.executeTakeFirst();
if (mek?.version !== params.mekVersion) {
throw new IntegrityError("Inactive MEK version");
}
const { categoryId } = await trx
.insertInto("category")
.values({
parent_id: params.parentId !== "root" ? params.parentId : null,
user_id: params.userId,
master_encryption_key_version: params.mekVersion,
encrypted_data_encryption_key: params.encDek,
data_encryption_key_version: params.dekVersion,
encrypted_name: params.encName,
})
.returning("id as categoryId")
.executeTakeFirstOrThrow();
await trx
.insertInto("category_log")
.values({
category_id: categoryId,
timestamp: new Date(),
action: "create",
new_name: params.encName,
})
.execute();
});
};
export const getAllCategoriesByParent = async (userId: number, parentId: CategoryId) => {
let query = db.selectFrom("category").selectAll().where("user_id", "=", userId);
query =
parentId === "root"
? query.where("parent_id", "is", null)
: query.where("parent_id", "=", parentId);
const categories = await query.execute();
return categories.map(
(category) =>
({
id: category.id,
parentId: category.parent_id ?? "root",
userId: category.user_id,
mekVersion: category.master_encryption_key_version,
encDek: category.encrypted_data_encryption_key,
dekVersion: category.data_encryption_key_version,
encName: category.encrypted_name,
}) satisfies Category,
);
};
export const getCategory = async (userId: number, categoryId: number) => {
const category = await db
.selectFrom("category")
.selectAll()
.where("id", "=", categoryId)
.where("user_id", "=", userId)
.limit(1)
.executeTakeFirst();
return category
? ({
id: category.id,
parentId: category.parent_id ?? "root",
userId: category.user_id,
mekVersion: category.master_encryption_key_version,
encDek: category.encrypted_data_encryption_key,
dekVersion: category.data_encryption_key_version,
encName: category.encrypted_name,
} satisfies Category)
: null;
};
export const setCategoryEncName = async (
userId: number,
categoryId: number,
dekVersion: Date,
encName: Ciphertext,
) => {
await db.transaction().execute(async (trx) => {
const category = await trx
.selectFrom("category")
.select("data_encryption_key_version")
.where("id", "=", categoryId)
.where("user_id", "=", userId)
.limit(1)
.forUpdate()
.executeTakeFirst();
if (!category) {
throw new IntegrityError("Category not found");
} else if (category.data_encryption_key_version.getTime() !== dekVersion.getTime()) {
throw new IntegrityError("Invalid DEK version");
}
await trx
.updateTable("category")
.set({ encrypted_name: encName })
.where("id", "=", categoryId)
.where("user_id", "=", userId)
.execute();
await trx
.insertInto("category_log")
.values({
category_id: categoryId,
timestamp: new Date(),
action: "rename",
new_name: encName,
})
.execute();
});
};
export const unregisterCategory = async (userId: number, categoryId: number) => {
const res = await db
.deleteFrom("category")
.where("id", "=", categoryId)
.where("user_id", "=", userId)
.executeTakeFirst();
if (res.numDeletedRows === 0n) {
throw new IntegrityError("Category not found");
}
};
-197
View File
@@ -1,197 +0,0 @@
import pg from "pg";
import { IntegrityError } from "./error";
import db from "./kysely";
import type { UserClientState } from "./schema";
interface Client {
id: number;
encPubKey: string;
sigPubKey: string;
}
interface UserClient {
userId: number;
clientId: number;
state: UserClientState;
}
interface UserClientWithDetails extends UserClient {
encPubKey: string;
sigPubKey: string;
}
export const createClient = async (encPubKey: string, sigPubKey: string, userId: number) => {
return await db
.transaction()
.setIsolationLevel("serializable")
.execute(async (trx) => {
const client = await trx
.selectFrom("client")
.where((eb) =>
eb.or([
eb("encryption_public_key", "=", encPubKey),
eb("encryption_public_key", "=", sigPubKey),
eb("signature_public_key", "=", encPubKey),
eb("signature_public_key", "=", sigPubKey),
]),
)
.limit(1)
.executeTakeFirst();
if (client) {
throw new IntegrityError("Public key(s) already registered");
}
const { clientId } = await trx
.insertInto("client")
.values({ encryption_public_key: encPubKey, signature_public_key: sigPubKey })
.returning("id as clientId")
.executeTakeFirstOrThrow();
await trx
.insertInto("user_client")
.values({ user_id: userId, client_id: clientId })
.execute();
return { id: clientId };
});
};
export const getClient = async (clientId: number) => {
const client = await db
.selectFrom("client")
.selectAll()
.where("id", "=", clientId)
.limit(1)
.executeTakeFirst();
return client
? ({
id: client.id,
encPubKey: client.encryption_public_key,
sigPubKey: client.signature_public_key,
} satisfies Client)
: null;
};
export const getClientByPubKeys = async (encPubKey: string, sigPubKey: string) => {
const client = await db
.selectFrom("client")
.selectAll()
.where("encryption_public_key", "=", encPubKey)
.where("signature_public_key", "=", sigPubKey)
.limit(1)
.executeTakeFirst();
return client
? ({
id: client.id,
encPubKey: client.encryption_public_key,
sigPubKey: client.signature_public_key,
} satisfies Client)
: null;
};
export const createUserClient = async (userId: number, clientId: number) => {
try {
await db.insertInto("user_client").values({ user_id: userId, client_id: clientId }).execute();
} catch (e) {
if (e instanceof pg.DatabaseError && e.code === "23505") {
throw new IntegrityError("User client already exists");
}
throw e;
}
};
export const getUserClient = async (userId: number, clientId: number) => {
const userClient = await db
.selectFrom("user_client")
.selectAll()
.where("user_id", "=", userId)
.where("client_id", "=", clientId)
.limit(1)
.executeTakeFirst();
return userClient
? ({
userId: userClient.user_id,
clientId: userClient.client_id,
state: userClient.state,
} satisfies UserClient)
: null;
};
export const getUserClientWithDetails = async (userId: number, clientId: number) => {
const userClient = await db
.selectFrom("user_client")
.innerJoin("client", "user_client.client_id", "client.id")
.selectAll()
.where("user_id", "=", userId)
.where("client_id", "=", clientId)
.limit(1)
.executeTakeFirst();
return userClient
? ({
userId: userClient.user_id,
clientId: userClient.client_id,
state: userClient.state,
encPubKey: userClient.encryption_public_key,
sigPubKey: userClient.signature_public_key,
} satisfies UserClientWithDetails)
: null;
};
export const setUserClientStateToPending = async (userId: number, clientId: number) => {
await db
.updateTable("user_client")
.set({ state: "pending" })
.where("user_id", "=", userId)
.where("client_id", "=", clientId)
.where("state", "=", "challenging")
.execute();
};
export const setUserClientStateToActive = async (userId: number, clientId: number) => {
await db
.updateTable("user_client")
.set({ state: "active" })
.where("user_id", "=", userId)
.where("client_id", "=", clientId)
.where("state", "=", "pending")
.execute();
};
export const registerUserClientChallenge = async (
userId: number,
clientId: number,
answer: string,
allowedIp: string,
expiresAt: Date,
) => {
const { id } = await db
.insertInto("user_client_challenge")
.values({
user_id: userId,
client_id: clientId,
answer,
allowed_ip: allowedIp,
expires_at: expiresAt,
})
.returning("id")
.executeTakeFirstOrThrow();
return { id };
};
export const consumeUserClientChallenge = async (
challengeId: number,
userId: number,
ip: string,
) => {
const challenge = await db
.deleteFrom("user_client_challenge")
.where("id", "=", challengeId)
.where("user_id", "=", userId)
.where("allowed_ip", "=", ip)
.where("expires_at", ">", new Date())
.returning(["client_id", "answer"])
.executeTakeFirst();
return challenge ? { clientId: challenge.client_id, answer: challenge.answer } : null;
};
export const cleanupExpiredUserClientChallenges = async () => {
await db.deleteFrom("user_client_challenge").where("expires_at", "<=", new Date()).execute();
};
+22
View File
@@ -0,0 +1,22 @@
import { PostgresDialect } from "kysely";
import pg from "pg";
export interface DatabaseConfig {
host?: string;
port?: number;
user?: string;
password?: string;
name?: string;
}
export const createDialect = (config: DatabaseConfig) => {
return new PostgresDialect({
pool: new pg.Pool({
host: config.host,
port: config.port,
user: config.user,
password: config.password,
database: config.name,
}),
});
};
-30
View File
@@ -1,30 +0,0 @@
type IntegrityErrorMessages =
// Category
| "Category not found"
// Challenge
| "Challenge already registered"
// Client
| "Public key(s) already registered"
| "User client already exists"
// File
| "Directory not found"
| "File not found"
| "File not found in category"
| "File already added to category"
| "Invalid DEK version"
// HSK
| "HSK already registered"
| "Inactive HSK version"
// MEK
| "MEK already registered"
| "Inactive MEK version"
// Session
| "Session not found"
| "Session already exists";
export class IntegrityError extends Error {
constructor(public message: IntegrityErrorMessages) {
super(message);
this.name = "IntegrityError";
}
}
Loaded 100 of 394 files, more files were not shown because too many files have changed in this diff. Show more