Uploading Files with Deno

In this workshop, we will break down how to programmatically upload files using Deno and the fetch API. We start with a very imperative approach with basic text, then we will upload an image file, add a JWT token, and finish with a functional example to add some safety.

One of the best parts of Deno is that it supports the browser API, so you get things like fetch, File, and FormData as a part of the global scope.

What will I discover by taking this tutorial?

• use Deno to upload files
• File object API
• FormData object API

What do I need to have/know?

• Git and have a Github Account
• Javascript
• Deno

Setup

We are going to use gitpod.io, so most of the setup is done, all you need is an internet connection, a browser, and a Github account.

Create a new repo using the Deno-starter template

1. Go to https://github.com/twilson63/deno-starter
2. Click use this template and name a new repo deno-upload-workshop or whatever you want
3. Launch gitpod.io by prefixing the new repo URL with https://gitpod.io/#

ex. if your repo name is https://github.com/twilson63/deno-upload-workshop — then in the browser address bar you would replace with
https://gitpod.io/#https://github.com/twilson63/deno-upload-workshop

This should start your Deno workspace on Gitpod.

Create a test webserver

In order to test our upload code, we need a webserver to receive our files and let us know that they were successfully uploaded.

Create a new file in your workspace called server.js

import { opine } from 'https://deno.land/x/opine@1.6.0/mod.ts'
import upload from './upload.js'

const app = opine()

app.post('/upload', upload('file'), async function (req, res) {
  //console.log('token', req.headers.get('authorization'))
  console.log('file', req.file.filename)
  const decoder = new TextDecoder()
  console.log(decoder.decode(req.file.content))

  res.send({ok: true})
})

app.get('/', function(req, res) {
  res.send('Hello')
})

app.listen(3000, () => console.log('listening on 3000'))

Create a new file in your workspace called upload.js

import * as R from 'https://cdn.skypack.dev/ramda@^0.27.1'
import { exists } from 'https://deno.land/std@0.102.0/fs/exists.ts'
import { MultipartReader } from "https://deno.land/std@0.102.0/mime/mod.ts";

const { compose, nth, split } = R
const TMP_DIR = '/tmp/hyper/uploads'

const getBoundary = compose(
  nth(1),
  split('='),
  nth(1),
  split(';')
)

export default function (fieldName = "file") {
  return async (req, res, next) => {
    let boundary;

    const contentType = req.get('content-type')
 
    if (contentType.startsWith('multipart/form-data')) {
      boundary = getBoundary(contentType);
    }
 
    if(!(await exists(TMP_DIR))) {
      await Deno.mkdir(TMP_DIR, { recursive: true })
    }

    const form = await new MultipartReader(req.body, boundary).readForm({
      maxMemory: 10 << 20,
      dir: TMP_DIR
    })
    
    req.file = form.files(fieldName)[0]

    next()

  }
}

Create a new terminal window and run your test server:

deno run -A server.js

Great! You have a server running that supports file uploads, I am not going to spend time talking about the server, as the focus of this workshop is to talk about the client. The server uses the opine the framework which is a port of the popular expressjs framework from NodeJS. And the upload.js file contains a middleware module that uses MultipartReader to ingest the multipart form from the request and attach the uploaded file to the req.file prop.

Take the time to push changes to github

* git add .
* git commit -am "0-created-upload-server"
* git push origin main

Basic File Upload

One of the best features of Deno is its dedication to implement and maintain browser APIs, in this workshop, we are going to use three browser APIs. fetch, File, and FormData. These APIs will give us the ability to send a file to our web server.

Create a new file called demo.js

In this file, let’s create a text file to upload using the File API

For more info about the File API — https://developer.mozilla.org/en-US/docs/Web/API/File

const file = new File(['hello world'], 'hello.txt')

Now that we have our file, let’s create a new form-data object and append our file as a field on the form-data object.

For more info about the FormData API — https://developer.mozilla.org/en-US/docs/Web/API/FormData

const form = new FormData()
form.append('file', file)

Upload the form using the fetch API

For more info about the fetch API — https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API

const res = await fetch('http://localhost:3000/upload', {
  method: 'POST',
  body: form
})

To complete the demo let’s print out the response from the request.

const txt = await res.text()
console.log(txt)

Here is what the whole demo.js file looks like:

const file = new File(['hello world'], 'hello.txt')
const form = new FormData()
form.append('file', file)

const res = await fetch('http://localhost:3000/upload', {
  method: 'POST',
  body: form
})

const txt = await res.text()
console.log(txt)

Make sure your test web server is running in a separate terminal: deno run -A server.js

Run your demo.js file

deno run --allow-net demo.js

You should see {ok: true} as the output of your client and over on the server you should see the filename and file contents.

🎉 Congrats! You uploaded a file using Deno!

Using the File API, we created a new text file, then we appended it to a form-data object and we added that object to the body property of the fetch options object. Some may point out, you could just add the file to the body of the fetch, and that may work for some server environments, but not all, if you have the ability to customize your server, you may want to go that route. But you do not have the option to attach additional fields to the upload process. For example, what if you wanted to place the field in a specific folder. With the form-data object, you can create a folder field and specify the folder you would like to place the uploaded file. In summary, the basic steps for uploading a text file to a server are a result of three steps, create a file object, append to a form-data object and provide the form-data object as the body of a fetch request that is being posted to the server.

Next, we will look at how to upload an image to a file server using Deno!

Take the time to commit the changes to your repository and push to github

* git add .
* git commit -am "1-upload-text-file"
* git push origin main

Image File Upload

Uploading binary files mainly image files can be a bit of a pain, it is certainly not as easy as uploading text files, but with Deno, the pattern is remarkably similar and simple.

Let’s create a new file called demo2.js in the project root directory.

In demo2.js let’s read a file using Deno

const f = await Deno.readFile('./road.png')

Now let’s upload the file just like we did with the text file:

const file = new File(f, 'road.png')
const form = new FormData()
form.append('file', file)

const res = await fetch('http://localhost:3000/upload', {
  method: 'POST',
  body: form
})

const txt = await res.text()
console.log(txt)

Run it!

deno run --allow-net --allow-read demo2.js

It will take some time to run, but eventually, it should upload the image file to the server.

In Deno, there is no difference between uploading an image or binary file and a text file, you just need to read the file into a buffer and then send it to the server, if the file is larger than a few megabytes you may want to stream its server, but that would be another workshop. 🙂

Add a JWT

Oftentimes, you may want to upload files to a secure endpoint, being able to send files to a secure endpoint is exactly the same as sending JSON documents to a secure endpoint, you will need to supply a JWT Token in the Authorization header.

Let’s create a new file called demo3.js - in this demo, we will send an HTML file to the server, but we will add a JWT as part of the request header.

import { create as jwt } from 'https://deno.land/x/djwt@v2.2/mod.ts'

const html = `<!doctype html>
<html>
  <head><title>Hello World</title></head>
  <body><h1>Hello World</h1></body>
</html>`

const f = new File([html], 'hello.html')
const form = new FormData()
form.append('file', f)

const token = await jwt({alg: 'HS256', typ: 'JWT'}, { sub: 'Hello'}, 'SECRET')

const res = await fetch('http://localhost:3000/upload', {
  method: 'POST', 
  headers: {
    authorization: `Bearer ${token}`
  },
  body: form
})

const txt = await res.text()

console.log(txt)

In this example, we import the JWT create method and then create a JWT token by calling jwt(header, payload, secret). We use this token to pass in the request as the Bearer token in the authorization header.

FPJS File Upload

In this section of the workshop, let’s build a more functional implementation of this upload process. Why? To create a safe flow, a safe flow is the process that catches any unexpected errors and makes sure to pass them through the flow to the error fork of the pipeline.

In this function javascript example, we will leverage crocks and ramda.

Let’s create a new file called demo4.js

import { crocks, R } from './deps.js'
import { create as jwtCreate } from 'https://deno.land/x/djwt@v2.2/mod.ts'

const { tryCatch, Result, Async, resultToAsync } = crocks
const { assoc, merge } = R
const { of } = Result

import some functional libraries to give some helpers to wrap potential side effects or error exception function calls.

const createFile = tryCatch(ctx => merge({
  file: new File([ctx.html], 'hello.html')
}, ctx))

const createFormData = tryCatch(ctx => {
  const data = new FormData()
  data.append('file', ctx.file)
  return merge({data}, ctx)
})

const jwt = (payload, secret) => Async.fromPromise(jwtCreate)({
  alg: 'HS256',
  typ: 'JWT'
}, payload, secret)

const asyncFetch = Async.fromPromise(fetch)

Using the tryCatch we can wrap these potential exception generating function calls using the Result ADT. If an exception occurs an Error Result will be passed, if not exception an Ok Result will be passed. We can create a Result ADT Pipeline.

const getResult = (html) => of({html})
  .chain(createFile)
  .chain(createFormData)

and an Async pipeline

const pipeline = result => resultToAsync(result)
  .chain(
    ctx => jwt({sub: 'foo'}, 'SECRET')
      .map(token => ({token, ...ctx}))
  )
  .chain(
    ctx => asyncFetch('http://localhost:3000/upload', {
      method: 'POST',
      headers: { authorization: `Bearer ${ctx.token}`},
      body: ctx.data
    })
  )
  .chain(
    res => Async.fromPromise(res.json.bind(res))()
  )

This completes the pure functions and now let’s jump to the implementation detail

// ---> implementation
// pure functions
const html = `<!doctype html>
<html>
  <head><title>Hello</title></head>
  <body><h1>Hello World</h1></body>
</html>`

pipeline(getResult(html)).fork(
  console.error, // handle errors
  console.log // handle success
)

Using this pipeline function with the fork method will invoke the file upload and either return an error or success.

deno run --allow-net demo4.js

Summary

In this workshop, we walk through the process of uploading files using Deno. The bottom line is that Deno is very much like uploading files in the browser, you can take advantage of native javascript browser APIs. fetch, File, and FormData, if you are worried that these APIs may generate unexpected exceptions, you can leverage functional javascript to create a pipeline that creates one exit point for errors and one exit point for successes.