Architecture
How the platform runs

Source file for diagrams

Overview

Whilst the processing side of the app is not particularly intensive, it makes sense to split this functionality away from the web server to avoid impacting performance of web app for user.
To do this we will use AWS technologies

Sequence creation

Sequence upload

For step2/3/4 read: Securing AWS S3 uploads using presigned URLs https://www.serverless.com/blog/s3-one-time-signed-url

Technical challenges for large filesizes

We use the AWS SDK functions to ensure large files have support for network failures (resumable uploads: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/welcome.html)
Upload in the browser poses a number of challenges.
In the case of video, it is due to file size (<6GB).
In the case of image, it is due to the amount (<=3000 images).
Files should be directly uploaded to S3, as they will require post processing after upload (using Lamda).
Stability and resume-ability of uploads are therefore important.
If upload abandoned or enters failed state, any uploaded content is deleted.

Image files

If any upload errors, we should attempt to retry 3 times, 1 minute after each failure.
If fails 3 times, upload considered failed. User shown error: "There was an error during upload. Please retry to create this Sequence. If this problem persists, please reduce the number of images in your Sequence."

Video files

Due to large sizes, videos should be chunked and uploaded in parallel.
If any upload chunk errors, we should attempt to retry 3 times, 1 minute after each failure.
If fails 3 times, upload considered failed. User shown error: "There was an error during upload. Please retry to create this Sequence. If this problem persists, please try a faster connection to upload this video."

Sequence processing

Jpg image upload processing

mp4 video upload processing

360 video upload processing

Sequence modification

For all other changes (e.g. delete images), these are changes to web DB (e.g. mark image as inactive).

Sequence publish (final state)

Sequence integrate

Last modified 1mo ago