deno_kv_fs

Deno KV file system, compatible with Deno deploy. Saves files in 64kb chunks. You can organize files into directories. You can control the KB/s rate for saving and reading files, useful for controlling uploads/downloads. Makes use of Web Streams API.

Contents

How to use

Instantiating the class:

const fs = new DenoKvFs();
const fs = new DenoKvFs(await Deno.openKv(/* your parameters */));

Remember that the use of space in the database must be optimized. When updating a file, it is first deleted. Only then will the new version of the file be saved. Files that are saved incompletely are automatically deleted.

The save method save is used to save files and has the following interface as input parameter:

interface SaveOptions {
  path: string[]; //Mandatory
  content: ReadableStream | Uint8Array; //Mandatory
  chunksPerSecond?: number; //Optional
  clientId?: string | undefined; //Optional
  maxFileSizeBytes?: number; //Optional
  allowedExtensions?: string[]; //Optional
}

The read, readDir, delete and deleteDir methods are intended to read and delete files, and have the following interface as input parameters:

interface ReadOptions {
  path: string[]; //Mandatory
  chunksPerSecond?: number; //Optional
  clientId?: string | undefined; //Optional
}

Examples

Saving data from a file

import { toReadableStream } from "jsr:@std/io";
const fileName = "myFile.txt";
let resData = await fs.save({
  path: ["my_dir", fileName],
  content: toReadableStream(await Deno.open(fileName)),
});

Saving data directly from bytes

Isso uses a Uint8Array as file content. This is not recommended, use only for internal resources of your application. For optimized use, use an instance of ReadableStream.

import { toReadableStream } from "jsr:@std/io";
const fileName = "myFile.txt";
let resData = await fs.save({
  path: ["my_dir", fileName],
  content: await Deno.readFile(fileName),
});

Saving data from a submitted form

const reqBody = await ctx.req.formData();
const existingFileNamesInTheUpload: { [key: string]: number } = {};
const res: any = {};
for (const item of reqBody.entries()) {
  if (item[1] instanceof File) {
    const formField: any = item[0];
    const fileData: any = item[1];
    if (!existingFileNamesInTheUpload[fileData.name]) {
      existingFileNamesInTheUpload[fileData.name] = 1;
    } else {
      existingFileNamesInTheUpload[fileData.name]++;
    }
    let prepend = "";
    if (existingFileNamesInTheUpload[fileData.name] > 1) {
      prepend += existingFileNamesInTheUpload[fileData.name].toString();
    }
    const fileName = prepend + fileData.name;
    let resData = await fs.save({
      path: ["my_dir", fileName],
      content: fileData.stream(),
    });
    if (res[formField] !== undefined) {
      if (Array.isArray(res[formField])) {
        res[formField].push(resData);
      } else {
        res[formField] = [res[formField], resData];
      }
    } else {
      res[formField] = resData;
    }
  }
}

Returning data

const fileName = "myFile.txt";
let resData = await fs.read({
  path: ["my_dir", fileName],
});
response.body = resData.content; //resData.content is an instance of ReadableStream

Returning data directly from bytes

This returns the file content as a Uint8Array. This is not recommended, use only for internal resources of your application. For optimized use, use the ReadableStream that comes by default.

const fileName = "myFile.txt";
let resData = await fs.read({
  path: ["my_dir", fileName],
});
response.body = await DenoKvFs.readStream(resData.content);

Example of a function to control data traffic

const existingRequests = fs.getClientReqs(loggedInUser.id);
const limit = (loggedInUser.isPremium() ? 100 : 10) / existingRequests;
let resData = await fs.read({
  path: ["my_dir", fileName],
  chunksPerSecond: limit,
  clientId: loggedInUser.id,
});
response.body = await DenoKvFs.readStream(resData.content);

Useful procedures included

  • static async readStream(stream: ReadableStream): Promise<Uint8Array>.
  • getClientReqs(clientId: string): number.
  • getAllFilesStatuses(): any[].
  • static async *pagedListIterator(listParams: any[], kv: Deno.Kv)
  • pathToURIComponent(path: string[]): string
  • URIComponentToPath(path: string): string[]
  • async save(options: SaveOptions): Promise<any>
  • async read(options: ReadOptions): Promise<any>
  • async readDir(options: ReadOptions): Promise<any[]>
  • async dirSize(options: ReadOptions): Promise<number>
  • async delete(options: ReadOptions): Promise<any>
  • deleteDir(options: ReadOptions): Promise<any[]>

All imports

import {
  DenoKvFs,
  ReadOptions,
  SaveOptions,
} from "https://deno.land/x/deno_kv_fs/mod.ts";

About

Author: Henrique Emanoel Viana, a Brazilian computer scientist, enthusiast of web technologies, cel: +55 (41) 99999-4664. URL: https://sites.google.com/view/henriqueviana

Improvements and suggestions are welcome!