deno_kv_fs

Deno KV file system, compatible with Deno deploy. Saves files in 64kb chunks. You can organize files into directories. You can control the KB/s rate for saving and reading files, useful for controlling uploads/downloads. Makes use of Web Streams API.

Contents

How to use

Instantiating the class:

const fs = new DenoKvFs();
const fs = new DenoKvFs(await Deno.openKv(/* your parameters */));

Remember that the use of space in the database must be optimized. When updating a file, it is first deleted. Only then will the new version of the file be saved. Files that are saved incompletely are automatically deleted.

Read methods return the processing status of a file (if it is currently being processed). This is useful for knowing the progress status of a save/update/delete. If a file does not exist, null is returned.

The method save is used to save files and has the following interface as input parameter:

interface SaveOptions {
  path: string[]; //Mandatory
  content: ReadableStream | Uint8Array; //Mandatory
  chunksPerSecond?: number; //Optional
  clientId?: string | undefined; //Optional
  maxFileSizeBytes?: number; //Optional
  allowedExtensions?: string[]; //Optional
}

The read, readDir, delete and deleteDir methods are intended to read and delete files, and have the following interface as input parameters:

interface ReadOptions {
  path: string[]; //Mandatory
  chunksPerSecond?: number; //Optional
  clientId?: string | undefined; //Optional
}

Examples

Saving data from a file

import { toReadableStream } from "jsr:@std/io";
const fileName = "myFile.txt";
let resData = await fs.save({
  path: ["my_dir", fileName],
  content: toReadableStream(await Deno.open(fileName)),
});

Saving data directly from bytes

Isso uses a Uint8Array as file content. This is not recommended, use only for internal resources of your application. For optimized use, use an instance of ReadableStream.

import { toReadableStream } from "jsr:@std/io";
const fileName = "myFile.txt";
let resData = await fs.save({
  path: ["my_dir", fileName],
  content: await Deno.readFile(fileName),
});

Saving data from a submitted form

const reqBody = await request.formData();
const existingFileNamesInTheUpload: { [key: string]: number } = {};
const res: any = {};
for (const item of reqBody.entries()) {
  if (item[1] instanceof File) {
    const formField: any = item[0];
    const fileData: any = item[1];
    if (!existingFileNamesInTheUpload[fileData.name]) {
      existingFileNamesInTheUpload[fileData.name] = 1;
    } else {
      existingFileNamesInTheUpload[fileData.name]++;
    }
    let prepend = "";
    if (existingFileNamesInTheUpload[fileData.name] > 1) {
      prepend += existingFileNamesInTheUpload[fileData.name].toString();
    }
    const fileName = prepend + fileData.name;
    let resData = await fs.save({
      path: ["my_dir", fileName],
      content: fileData.stream(),
    });
    if (res[formField] !== undefined) {
      if (Array.isArray(res[formField])) {
        res[formField].push(resData);
      } else {
        res[formField] = [res[formField], resData];
      }
    } else {
      res[formField] = resData;
    }
  }
}

In frontend

<form id="yourFormId" enctype="multipart/form-data" action="/upload" method="post">
  <input type="file" name="file1" multiple><br>
  <input type="submit" value="Submit">
</form>
<script>
  var files = document.querySelector("#yourFormId input[type=file]").files;
  var name = document.querySelector("#yourFormId input[type=file]").getAttribute(
    "name",
  );
  var form = new FormData();
  for (var i = 0; i < files.length; i++) {
    form.append(`${name}_${i}`, files[i]);
  }
  var res = await fetch(`/your_POST_URL`, { //Fetch API automatically puts the form in the format "multipart/form-data".
    method: "POST",
    body: form,
  }).then((response) => response.json());
  console.log(res);
</script>

Returning data

const fileName = "myFile.txt";
let resData = await fs.read({
  path: ["my_dir", fileName],
});
response.body = resData.content; //resData.content is an instance of ReadableStream

Returning data directly from bytes

This returns the file content as a Uint8Array. This is not recommended, use only for internal resources of your application. For optimized use, use the ReadableStream that comes by default.

const fileName = "myFile.txt";
let resData = await fs.read({
  path: ["my_dir", fileName],
});
response.body = await DenoKvFs.readStream(resData.content);

Example of a function to control data traffic

const existingRequests = fs.getClientReqs(loggedInUser.id); //Use the same as clientId
const limit = (loggedInUser.isPremium() ? 100 : 10) / existingRequests;
let resData = await fs.read({
  path: ["my_dir", fileName],
  chunksPerSecond: limit,
  clientId: loggedInUser.id, //The clientId can also be the remote address of a request, for example.
});
response.body = resData.content;

Useful procedures included

  • static async readStream(stream: ReadableStream): Promise<Uint8Array>
  • static async *pagedListIterator(listParams: any[], kv: Deno.Kv)
  • getClientReqs(clientId: string): number
  • getAllFilesStatuses(): any[]
  • pathToURIComponent(path: string[]): string
  • URIComponentToPath(path: string): string[]
  • async save(options: SaveOptions): Promise<any>
  • async read(options: ReadOptions): Promise<any>
  • async readDir(options: ReadOptions): Promise<any[]>
  • async dirSize(options: ReadOptions): Promise<number>
  • async delete(options: ReadOptions): Promise<any>
  • async deleteDir(options: ReadOptions): Promise<any[]>

All imports

import {
  DenoKvFs,
  ReadOptions,
  SaveOptions,
} from "https://deno.land/x/deno_kv_fs/mod.ts";

About

Author: Henrique Emanoel Viana, a Brazilian computer scientist, enthusiast of web technologies, cel: +55 (41) 99999-4664. URL: https://sites.google.com/view/henriqueviana

Improvements and suggestions are welcome!